Python Bytes: #265 Get asizeof pympler and muppy

<p><strong>Watch the live stream:</strong></p> <a href='https://www.youtube.com/watch?v=qjQwrS0xLqI' style='font-weight: bold;'>Watch on YouTube</a><br> <br> <p><strong>About the show</strong></p> <p>Sponsored by <strong>us:</strong></p> <ul> <li>Check out the <a href="https://training.talkpython.fm/courses/all"><strong>courses over at Talk Python</strong></a></li> <li>And <a href="https://pythontest.com/pytest-book/"><strong>Brian’s book too</strong></a>!</li> </ul> <p>Special guest: Matt Kramer (<a href="https://twitter.com/__matt_kramer__">@__matt_kramer__</a>)</p> <p><strong>Michael #1: Survey results</strong></p> <ul> <li>Question 1: <img src="https://ift.tt/3HCHKpE" alt="" /></li> </ul> <p>Question 2:</p> <p><img src="https://ift.tt/32L3LE5" alt="" /></p> <ul> <li>In terms of too long, the “extras” section has started at these times in the last 4 episodes: <ul> <li>39m, 32m, 35m, and 33m ~= 34m on average </li> </ul></li> </ul> <p><strong>Brian #2:</strong> <strong>Modern attrs API</strong></p> <ul> <li><a href="https://www.attrs.org/en/latest/overview.html">attrs overview</a> now focus on using <code>@define</code></li> <li>History of attrs article: <a href="https://hynek.me/articles/import-attrs/">import attrs</a>, by Hynek <ul> <li>predecessor was called <code>characteristic</code>. </li> <li>A discussion between Glyph and Hynek in 2015 about where to take the idea.</li> <li><code>attrs</code> popularity takes off in 2016 after a post by Glyph: <a href="https://glyph.twistedmatrix.com/2016/08/attrs.html"><em>‌The One Python Library Everyone Needs</em></a></li> <li>In 2017 people started wanting something like attrs in std library. Thus PEP 557 and dataclasses. Hynek, Eric Smith, and Guido discuss it at PyCon US 2017. </li> <li>dataclasses, with a subset of attrs functionality, was introduced in Python 3.7. </li> <li>Types take off. attrs starts supporting type hints as well, even before Python 3.7</li> <li>Post 3.7, some people start wondering if they still need attrs, since they have dataclasses.</li> <li><code>@define</code>, <code>field()</code> and other API improvements came with attrs 20.1.0 in 2020.</li> <li>attrs 21.3.0 released in December, with what Hynek calls “Modern attrs”.</li> </ul></li> <li>OG attrs:</li> </ul> <div class="codehilite"><pre><span></span><code><span class="kn">import</span> <span class="nn">attr</span> <span class="nd">@attr</span><span class="o">.</span><span class="n">s</span> <span class="k">class</span> <span class="nc">Point</span><span class="p">:</span> <span class="n">x</span> <span class="o">=</span> <span class="n">attr</span><span class="o">.</span><span class="n">ib</span><span class="p">()</span> <span class="n">y</span> <span class="o">=</span> <span class="n">attr</span><span class="o">.</span><span class="n">ib</span><span class="p">()</span> </code></pre></div> <ul> <li>modern attrs:</li> </ul> <div class="codehilite"><pre><span></span><code><span class="kn">from</span> <span class="nn">attr</span> <span class="kn">import</span> <span class="n">define</span> <span class="nd">@define</span> <span class="k">class</span> <span class="nc">Point</span><span class="p">:</span> <span class="n">x</span><span class="p">:</span> <span class="nb">int</span> <span class="n">y</span><span class="p">:</span> <span class="nb">int</span> </code></pre></div> <ul> <li>Many reasons to use attrs listed in <a href="https://www.attrs.org/en/latest/why.html">Why not…</a>, which is an excellent read. <ul> <li>why not dataclasses?</li> <li>less powerful than attrs, intentionally <ul> <li>attrs has validators, converters, equality customization, …</li> </ul></li> <li>attrs doesn’t force type annotation if you don’t like them</li> <li>slots on by default, dataclasses only support slots in Python 3.10 and are off by default <ul> <li>attrs can and will move faster</li> </ul></li> <li>See also comparisons with pydantic, named tuples, tuples, dicts, hand-written classes</li> </ul></li> </ul> <p><strong>Matt</strong> <strong>#3:</strong> <a href="https://craftinginterpreters.com/"><strong>Crafting Interpreters</strong></a></p> <ul> <li>Wanting to learn more about how Python works “under the hood”, I first read Anthony Shaw’s <a href="https://realpython.com/products/cpython-internals-book/">CPython internals</a> book <ul> <li>A fantastic, detailed overview of how CPython is implemented</li> </ul></li> <li>Since I don’t have a formal CS background, I found myself wanting to learn a bit more about the fundamentals <ul> <li>Parsing, Tokenization, Bytecode, data structures, etc.</li> </ul></li> <li>Crafting Interpreters is an incredible book by Bob Nystrom (on Dart team at Google)</li> <li>Although not Python, you walk through the implementation of a dynamic, interpreted language from scratch</li> <li>Implement same language (called lox) in two interpreters <ul> <li>First a direct evaluation of Abstract Syntax Tree, written in Java</li> <li>Second is a bytecode interpreter, written from the ground up in C, including a compiler</li> </ul></li> <li>Every line of code is in the book, it is incredibly well-written and beautifully rendered</li> <li>I highly recommend to anyone wanting to learn more about language design &amp; implementation</li> </ul> <p><strong>Michael #4:</strong> <a href="https://github.com/23andMe/Yamale"><strong>Yamele - A schema and validator for YAML</strong></a></p> <ul> <li>via Andrew Simon</li> <li>A basic schema:</li> </ul> <pre><code>name: str() age: int(max=200) height: num() awesome: bool() </code></pre> <ul> <li>And some YAML that validates:</li> </ul> <pre><code>name: Bill age: 26 height: 6.2 awesome: True </code></pre> <ul> <li>Take a look at the <a href="https://github.com/23andMe/Yamale#examples">Examples</a> section for more complex schema ideas.</li> <li>⚠️ Ensure that your schema definitions come from internal or trusted sources. Yamale does not protect against intentionally malicious schemas.</li> </ul> <p><strong>Brian #5:</strong> <a href="https://pympler.readthedocs.io/en/latest/"><strong>pympler</strong></a></p> <ul> <li>Inspired by something Bob Belderbos wrote about sizes of objects, I think.</li> <li>“Pympler is a development tool to measure, monitor and analyze the memory behavior of Python objects in a running Python application.</li> <li>By pympling a Python application, detailed insight in the size and the lifetime of Python objects can be obtained. Undesirable or unexpected runtime behavior like memory bloat and other “pymples” can easily be identified.”</li> <li>3 separate modules for profiling <ul> <li><a href="https://pympler.readthedocs.io/en/latest/asizeof.html#asizeof">asizeof</a> module provides basic size information for one or several Python objects</li> <li><a href="https://pympler.readthedocs.io/en/latest/muppy.html#muppy">muppy</a> is used for on-line monitoring of a Python application</li> <li><a href="https://pympler.readthedocs.io/en/latest/classtracker.html#classtracker">Class Tracker</a> provides off-line analysis of the lifetime of selected Python objects.</li> </ul></li> <li>asizeof is what I looked at recently <ul> <li>In contrast to <code>sys.getsizeof</code>, <code>asizeof</code> sizes objects recursively. </li> <li>You can use one of the <a href="https://pympler.readthedocs.io/en/latest/asizeof.html#asizeof">asizeof</a> functions to get the size of these objects and all associated referents:</li> </ul></li> </ul> <pre><code>&gt;&gt;&gt; from pympler import asizeof &gt;&gt;&gt; obj = [1, 2, (3, 4), 'text'] &gt;&gt;&gt; asizeof.asizeof(obj) 176 &gt;&gt;&gt; print(asizeof.asized(obj, detail=1).format()) [1, 2, (3, 4), 'text'] size=176 flat=48 (3, 4) size=64 flat=32 'text' size=32 flat=32 1 size=16 flat=16 2 size=16 flat=16 </code></pre> <ul> <li>“Function <strong>flatsize</strong> returns the <em>flat size</em> of a Python object in bytes defined as the <em>basic size</em> plus the <em>item size</em> times the <em>length</em> of the given object.”</li> </ul> <p><strong>Matt</strong> <strong>#6:</strong> <a href="https://hvplot.holoviz.org/user_guide/Interactive.html#"><strong>hvPlot Interactive</strong></a></p> <ul> <li>hvPlot is a high-level plotting API that is part of the PyData ecosystem, built on HoloViews</li> <li>My colleague Phillip Rudiger recently gave a talk at PyData Global on a new <code>.interactive</code> feature</li> <li><a href="https://discourse.holoviz.org/t/pydata-2021-build-polished-data-driven-applications-directly-from-your-pandas-or-xarray-pipelines/3017">Here’s an announcement in the HoloViz forum</a></li> <li>Allows integration of widgets directly into <code>pandas</code> analysis pipeline (method-chain), so you can add interactivity to your notebook for exploratory data analysis, or serve it as a <a href="https://panel.holoviz.org/">Panel app</a></li> <li><a href="https://gist.github.com/MarcSkovMadsen/ffb273636dced88705c8c88d5ee28f23">Gist &amp; video</a> by Marc Skov Madsen</li> </ul> <p><strong>Extras</strong> </p> <p>Michael:</p> <ul> <li><a href="https://typora.io/"><strong>Typora app</strong></a>, recommended!</li> <li><a href="https://twitter.com/willmcgugan/status/1474342602665304076?s=12"><strong>Congrats Will</strong></a></li> <li>Got a chance to solve a race condition with <a href="https://tenacity.readthedocs.io/en/latest/"><strong>Tenacity</strong></a></li> <li>New <a href="https://docs.github.com/en/issues/trying-out-the-new-projects-experience"><strong>project management at GitHub</strong></a></li> </ul> <p>Matt: </p> <ul> <li>Check out new <a href="https://community.anaconda.cloud/">Anaconda Nucleus Community</a> forums!</li> <li>We’re hiring, and remote-first. Check out <a href="http://anaconda.com/careers">anaconda.com/careers</a></li> <li><a href="https://blog.pyston.org/2021/12/07/pre-compiled-packages-now-available-for-pyston/">Pre-compiled packages now available for Pyston</a></li> <li>We have an upcoming webinar from Martin Durant: <a href="https://ift.tt/3t5UO2K Your Big Problem is I/O Bound</a></li> </ul> <p><strong>Joke:</strong> </p> <p><img src="https://ift.tt/32LuEYA" alt="" /></p>

from Planet Python
via read more