Introducing Facebook and Instagram PyCon 2019 Principal Sponsors!
Facebook and Instagram have been big supporters of the Python language through sponsored development of PyTorch and Pyre. This year the PSF is proud to have Facebook and Instagram as Principal Sponsors of our foundation and PyCon 2019 in Cleveland. They want to share the variety of ways they're not only using Python but how they’ve implemented it in creating, teaching, and developing their programs. We spoke with them to find out more about their recent efforts in the Python community, and here’s what they had to share regarding the integration of Python in their systems by team.
Facebook Production Engineering
There are engineers within Facebook that are 100% focused on maintaining the Python infrastructure of the other developers that use Python.Python in Production Engineering Jan 2019.pdf
There's a whole category of software that we write at Facebook that's basically like the glue that holds all the other software together. So this is like tools that help people operate their production services and configuration for stuff. So it's like the stuff that isn't like the core service, but it helps you operate and it helps you work better with other services in the same ecosystem. The vast majority of that glue (code) is written in Python, and we do that because it's not code for which performance is absolutely paramount because typically a user is using some tool or we're doing some kind of configuration - it's not like doing some heavy computation - but Python is so easy to use and easy to change and really flexible so we like using it for that purpose, because it helps us write good tools really quickly.
Python helps automate operational tasks.
Instagram talks at PyCon this year:
Releasing the World's Largest Python Site Every 7 Minutes
How to JIT: Writing a Python JIT from scratch in pure Python
Relevant Blogs:
https://thenewstack.io/instagram-makes-smooth-move-python-3/https://instagram-engineering.com/copy-on-write-friendly-python-garbage-collection-ad6ed5233ddf
https://instagram-engineering.com/profiling-cpython-at-instagram-89d4cbeeb898
- Kinds of problems we encounter:
- Release
PyTorch
PyTorch is a library for machine learning and deep learning specifically. The thing that makes PyTorch popular within the research community and also at Facebook is that it's very deeply integrated with the Python numerical computing ecosystem.https://research.fb.com/downloads/pytorch/
With PyTorch, the primary interface for users is in Python, but a lot of the internals are written in C++.
“There are several different types of people who use PyTorch. We have our research scientists in FAIR, and they use Python every day to prototype and experiment with different model architectures, with different strategies, and it lets them move extremely quickly and iterate rapidly because it's such a flexible base. So we will basically have people who will tweak their model, come up with a new way to structure it, and then run a bunch of experiments to see if it worked better on the datasets that they care about. So that's the day-to-day of someone who's doing research.”
“We have increasing usage of PyTorch in production for applications to improve our products, and the day-to-day there is that people are trying to take ideas that the researchers have determined are promising and try and get them into a form that is suitable for production, so it's like cleaning up the code, making it more efficient, that kind of thing - and then staging it, trying to integrate it with the product, seeing how the product metrics move up and down, that kind of thing.”
Academic research
Facebook and Instagram have built Pyre (https://www.pyre-check.org/), a typechecker for Python that is optimized for large codebases that change frequently. It is the default type checker for Instagram Sever code, as well as a large number of backend Python projects. Tools like Pyre make it easier and faster to build high quality products in Python.https://www.youtube.com/watch?v=hWV8t494N88 https://us.pycon.org/2019/schedule/presentation/148/
from Planet Python
via read more
No comments:
Post a Comment