Python Bytes: #174 Happy developers use Python 3

<p>Sponsored by us! <a href="https://training.talkpython.fm/"><strong>Talk Python courses</strong></a> &amp; <a href="https://pragprog.com/book/bopytest/python-testing-with-pytest"><strong>pytest book</strong></a>.</p> <p><strong>Topic #0: Quick chat about COVID 19.</strong></p> <p><strong>Brian #1:</strong> <a href="https://labs.quansight.org/blog/2020/03/documentation-as-a-way-to-build-community/"><strong>Documentation as a way to build Community</strong></a></p> <ul> <li>Melissa Mendonça</li> <li>“… educational materials can have a huge impact and effectively bring people into the community.”</li> <li>Quality documentation for OSS is often lacking due to: <ul> <li>decentralized development</li> <li>documentation is not as glamorous or as praised as new features or major bug fixes</li> <li>“Even when the community is welcoming, documentation is often seen as a "good first issue", meaning that the docs end up being written by the least experienced contributors in the community.”</li> </ul></li> <li>Possible solution: <ul> <li>organize/re-organize docs into: <ul> <li>tutorials</li> <li>how-tos</li> <li>reference guide</li> <li>explanations</li> </ul></li> <li>consequences: <ul> <li>Improving on the quality and discoverability </li> <li>Clear difference between docs aimed at different users </li> <li>Give users more opportunities to contribute, generating content that can be shared directly on the official documentation</li> <li>Building a documentation team as a first-class team in the project, which helps create an explicit role as <em>documentation creator</em>. This helps people better identify how they can contribute beyond code.</li> <li>Diversifying our contributor base, allowing people from different levels of expertise and different life experiences to contribute. This is also extremely important so that we have a better understanding of our community and can be accessible, unbiased and welcoming to all people.</li> </ul></li> </ul></li> <li>Referenced in article: <a href="https://www.divio.com/blog/documentation">"What nobody tells you about documentation"</a></li> </ul> <p><strong>Michael #2:</strong> <a href="https://openfolder.sh/django-faster-speed-tutorial"><strong>The Django Speed Handbook: making a Django app faster</strong></a></p> <ul> <li>By <a href="https://twitter.com/SHxKM">Shibel Mansour</a></li> <li>Speed of your app is very important: 100ms is an eternity. SEO, user conversions, bounce rates, etc.</li> <li>Use the tried-and-true <a href="https://github.com/jazzband/django-debug-toolbar"><code>django-debug-toolbar</code></a>. <ul> <li>Analyze your request/response cycles and see where most of the time is spent. </li> <li>Provides database query execution times and provides a nice SQL <code>EXPLAIN</code> in a separate pane that appears in the browser.</li> </ul></li> <li><strong>ORM/Database:</strong> Two ORM functionalities I want to mention first: these are <code>select_related</code> and <code>prefetch_related</code>. Nice 24x perf improvement example in the article. Basically, beware of the N+1 problem.</li> <li><strong>Indexes:</strong> Be sure to add them but they <em>slow</em> writes.</li> <li><strong>Pagination:</strong> Use it if you have lots of data</li> <li><strong>Async / background tasks</strong>.</li> <li><strong>Content size:</strong> Shrunk 9x by adding gzip middleware</li> <li><strong>Static files:</strong> minify and bundle as you can, cache, serve through nginx, etc. <ul> <li>At Python Bytes, Talk Python, etc, we use <code>webassets</code>, <code>cssmin</code>, and <code>jsmin</code>.</li> </ul></li> <li><strong>PageSpeed</strong> from Google, <a href="https://ift.tt/2UzuBXw python’s ranking</strong></a>.</li> <li><strong>ImageOptim</strong> (for <a href="https://imageoptim.com/mac">macOS</a>, <a href="https://imageoptim.com/versions">others</a>)</li> <li><strong>Lazy-loading images:</strong> Lazily loading images means that we only request them when or a little before they enter the client’s (user’s) viewport. With excellent, dependency-free JavaScript libraries like <a href="https://github.com/verlok/lazyload">LazyLoad</a>, there really isn’t an excuse to not lazy-load images. Moreover, Google Chrome natively supports the <code>lazy</code> attribute.</li> <li>Remember: Test and measure everything, before and after.</li> </ul> <p><strong>Brian #3:</strong> <a href="https://github.com/konradhalas/dacite"><strong>dacite: simplifies creation of data classes from dictionaries</strong></a></p> <ul> <li>Konrad Hałas</li> <li>dataclasses are awesome <ul> <li>quick and easy</li> <li>fields can <ul> <li>have default values</li> <li>be excluded from comparison and/or repr and more</li> </ul></li> </ul></li> <li>data often gets to us in dictionaries</li> <li>Converting from dict to dataclass is trivial for trivial cases: <code>x = MyClass(**data_as_dict)</code></li> <li>For more complicated conversions, you need <code>dacite</code></li> <li><code>dacite.from_dict</code> supports: <ul> <li>nested structures</li> <li>optional fields and unions</li> <li>collections</li> <li>type_hooks, which allow you to have custom converters for certain types</li> </ul></li> <li>strict mode. Normally allows extra input data that is just ignored if it doesn’t match up with fields. But you can use strict to not allow that.</li> <li>Raises exceptions when something weird happens, like the wrong type, missing values, etc.</li> </ul> <p><strong>Michael #4:</strong> <a href="https://engineering.linkedin.com/blog/2020/how-we-retired-python-2-and-improved-developer-happiness"><strong>How we retired Python 2 and improved developer happiness</strong></a></p> <ul> <li>By <a href="https://engineering.linkedin.com/blog/authors/b/barry-warsaw">Barry Warsaw</a></li> <li><a href="https://pythonclock.org/">The Python Clock</a> is at 0:00.</li> <li>In 2018, LinkedIn embarked on a multi-quarter effort to fully transition to a Python 3 code base.</li> <li>In total, the effort entailed the migration of about 550 code repositories.</li> <li>They don't use Python in our product or as a monolithic web service, and instead have hundreds of independent microservices and tools, and dozens of supporting libraries, all owned by independent teams in separate repositories.</li> <li>In the early days, most of internal libraries were ported to be “<a href="https://docs.python.org/3/howto/pyporting.html">bilingual</a>,” meaning they could be used in either Python 2 or 3.</li> <li>Given that the migration affected all of LinkedIn engineering across so many disparate teams and thousands of engineers, the effort was overseen by our Horizontal Initiatives (HI) program.</li> <li><strong>Phase 1:</strong> In the first quarter of 2019, we performed detailed dependency graphing, identifying a number of repositories that were more foundational, and thus needed to be fully ported first because they blocked the ports of everything that depended on them.</li> <li><strong>Phase 2:</strong> In the second quarter of 2019, we identified the remainder of repositories that needed porting</li> <li><strong>Post-migration reflections</strong>: Our primary indicator for completing the migration of a multiproduct was that it built successfully and passed its unit and integration tests.</li> <li>For other organizations planning or in the midst of their own migration paths, we offer the following guidelines: <ul> <li>Plan early, and engage your organization’s Python experts. Find and leverage champions in your affected teams, and promote the benefits of Python 3.</li> <li>Adopt the bilingual approach to supporting libraries so that consumers of your libraries can port to Python 3 on their own schedules.</li> <li>Invest in tests and code coverage—these will be your best success metrics.</li> <li>Ensure that your data models are explicit and clear, especially in identifying which data are bytes and which are human-readable text.</li> </ul></li> <li>Benefits: <ul> <li>No longer have to worry about supporting Python 2 and have seen our support loads decrease. </li> <li>Can now depend on the latest open source libraries and tools, and free from the constrictions of having to write bilingual Python. </li> <li>Opportunistically and enthusiastically adopting type hinting and the <a href="https://mypy.readthedocs.io/en/latest/">mypy</a> type checker, improving the overall quality, craft, and readability of Python code bases. </li> </ul></li> </ul> <p><strong>Brian #5:</strong> <a href="http://calpaterson.com/activerecord.html"><strong>The Troublesome Active Record Pattern</strong></a></p> <ul> <li>Cal Paterson</li> <li>"Object relational mappers" (ORMs) exist to bridge the gap between the programmers' friend (the object), and the database's primitive (the relation). </li> <li>Examples include Django ORM and SQLAlchemy</li> <li>The Active Record pattern of data access is marked by: <ol> <li>A whole-object basis</li> <li>Access by key (mostly primary key)</li> </ol></li> <li>Problem: Queries that don’t need all information for objects retrieve it all anyway, and it’s easy to code for loops to select or collect info that are wildly inefficient. <ul> <li>how many books are there</li> <li>how many books about software testing written by Oregon authors</li> </ul></li> <li>Problem: transactions. people can forget to use transactions, some ORMs don’t support them, they are not taught in beginner tutorials, etc. <ul> <li>SQLAlchemy has sessions</li> <li>Django has <code>atomic()</code></li> </ul></li> <li>REST APIs can suffer the same problems.</li> <li>Solutions: <ul> <li>just use SQL</li> <li>first class queries</li> <li>first class transactions</li> <li>avoid Active Record style access patterns </li> <li>Be careful with REST APIs <ul> <li>Alternatives: <ul> <li>GraphQL</li> <li>RPC-style APIs</li> </ul></li> </ul></li> </ul></li> </ul> <p><strong>Michael #6:</strong> <a href="https://blog.meadsteve.dev/programming/2020/02/10/types-at-the-edges-in-python/"><strong>Types at the edges in Python</strong></a></p> <ul> <li>By Steve Brazier</li> <li>For a new web service in python there are 3 things to start with: <ul> <li><a href="https://pydantic-docs.helpmanual.io/">Pydantic</a></li> <li><a href="http://mypy-lang.org/">mypy</a></li> <li>Production error tracking of some kind</li> </ul></li> <li>Why: Because what is this about? <code>AttributeError: 'NoneType' object has no attribute 'strip'</code> It should be: <code>none is not an allowed value (type=type_error.none.not_allowed)</code></li> <li>We then launch this code into production and our assumptions are tested against reality. If we’re lucky our assumptions turn out to be correct. If not we likely encounter some cryptic <code>NoneType</code> errors like the one at the start of this post.</li> <li>Pydantic can help by formalizing our assumptions.</li> <li><strong>mypy carries on helping</strong>: Once you see the error at the start of this post (thanks error reporting) you know what is wrong about assumptions. Make the following change to your code: <code>field: typing.Optional[str]</code></li> <li>BTW: FastAPI integrates with Pydantic out of the box.</li> <li>A mini-kata like exercise here that can be worked through: <a href="https://github.com/meadsteve/types-at-the-edges-minikata">meadsteve/types-at-the-edges-minikata</a></li> </ul> <p>Extras:</p> <p>Michael:</p> <ul> <li><a href="https://jackmckew.dev/python-bytes-awesome-package-list.html"><strong>Python Bytes Awesome Package List</strong></a> by Jack Mckew</li> <li><a href="https://flip.it/dp9cxY"><strong>Visual Basic Will Stall Out With .NET 5</strong></a></li> <li><a href="https://twitter.com/b33k33p/status/1240022365125443584"><strong>COVID 19 data sets</strong></a></li> <li>New course in dev: <strong>Adding a CMS to Your Data-Driven Web App</strong> [in Pyramid|Flask]</li> </ul> <p>Joke:</p> <p><img src="https://ift.tt/3axIg84" alt="" /></p> <p><a href="https://trello-attachments.s3.amazonaws.com/58e3f7c543422d7f3ad84f33/5e5ff59ebac11305019c191c/73ba3e752f0d132242e626d8ffc53cf2/docs.jpg">https://trello-attachments.s3.amazonaws.com/58e3f7c543422d7f3ad84f33/5e5ff59ebac11305019c191c/73ba3e752f0d132242e626d8ffc53cf2/docs.jpg</a></p>

from Planet Python
via read more