Python Bytes: #180 Transactional file IO with Python and safer

<p>Sponsored by <strong>DigitalOcean</strong>: <a href="https://pythonbytes.fm/digitalocean"><strong>pythonbytes.fm/digitalocean</strong></a> - $100 credit for new users to build something awesome.</p> <p><strong>Michael #1:</strong> <a href="https://lists.ubuntu.com/archives/ubuntu-announce/2020-April/000256.html"><strong>Ubuntu 20.04 is out</strong></a>!</p> <ul> <li>Next LTS support version since 26th April 2018 (18.04).</li> <li>Comes with Python 3.8 included!</li> <li>Already upgraded all our servers, super smooth.</li> <li>Kernel has been updated to the 5.4 based Linux kernel, with additional support for Wireguard VPN, AUFS5, and improved support for IBM, Intel, Raspberry Pi and AMD hardware.</li> <li>Features the latest version of the GNOME desktop environment.</li> <li>Brings support for installing an Ubuntu desktop system on top of ZFS.</li> <li>20.04 already an option on DigitalOcean ;)</li> </ul> <p><strong>Brian #2:</strong> <a href="https://lerner.co.il/2020/04/27/working-with-warnings-in-python/"><strong>Working with warnings in Python</strong></a></p> <ul> <li>(Or: When is an exception not an exception?)</li> <li>Reuven Lerner</li> <li>Exceptions, the class hierarchy of exceptions, and warnings.</li> <li>“… most of the time, warnings are aimed at developers rather than users. Warnings in Python are sort of like the “service needed” light on a car; the user might know that something is wrong, but only a qualified repairperson will know what to do. Developers should avoid showing warnings to end users.”</li> <li>Python’s warning system …: <ul> <li>It treats the warnings as a separate type of output, so that we cannot confuse it with either exceptions or the program’s printed text,</li> <li>It lets us indicate what kind of warning we’re sending the user,</li> <li>It lets the user indicate what should happen with different types of warnings, with some causing fatal errors, others displaying their messages on the screen, and still others being ignored,</li> <li>It lets programmers develop their own, new kinds of warnings.</li> </ul></li> <li>Reuven goes on to show how to use warnings in your code. <ul> <li>using them</li> <li>creating custom warnings</li> <li>filtering</li> </ul></li> </ul> <p><strong>Michael #3:</strong> <a href="https://medium.com/@TomSwirly/%EF%B8%8F-safer-a-safer-file-writer-%EF%B8%8F-5fe267dbe3f5"><strong>Safer file writer</strong></a></p> <ul> <li>pip installable, see <a href="https://medium.com/@TomSwirly/%EF%B8%8F-safer-a-safer-file-writer-%EF%B8%8F-5fe267dbe3f5">the article</a> and the <a href="https://github.com/rec/safer">repo too</a>.</li> <li>Consider this code:</li> </ul> <pre><code> with open(filename, 'w') as fp: json.dump(data, fp) </code></pre> <ul> <li>It’s using with, so it’s good right? </li> <li>Well the file itself may be overwritten and maybe corrupted</li> <li>With <code>safer</code>, you write almost identical code:</li> </ul> <pre><code>with safer.open(filename, 'w') as fp: json.dump(data, fp) </code></pre> <ul> <li>Now if <code>json.dump()</code> throws an exception, the original file is unchanged, so your important data file lives to see another day.</li> <li>The actual 28 lines of code is pretty interesting: <a href="https://github.com/rec/safer/blob/v1.0.0/safer.py#L70-L97">https://github.com/rec/safer/blob/v1.0.0/safer.py#L70-L97</a></li> </ul> <p><strong>Brian #4:</strong> <a href="https://github.com/codespell-project/codespell"><strong>codespell</strong></a></p> <ul> <li>codespell : Fix common misspellings in text files. It's designed primarily for checking misspelled words in source code, but it can be used with other files as well.</li> <li>I got a cool <a href="https://github.com/okken/cards/pull/43">pull request against the cards project</a> to add a pre-commit hook to run codespell. (Thanks Christian Clauss)</li> <li>codespell caught a documentation spelling error in cards, where I had spelled “arguments” as “arguements”. Oops.</li> <li>Spelling errors are annoying and embarrassing in code and comments, and distracting. Also hard to deal with using traditional spell checkers. So super glad this is a thing. </li> </ul> <p><strong>Michael #5:</strong> <a href="https://github.com/P403n1x87/austin"><strong>Austin profiler</strong></a></p> <ul> <li>via <a href="http://twitter.com/anthonypjshaw/">Anthony Shaw</a></li> <li>Python frame stack sampler for CPython</li> <li>Profiles CPU and Memory!</li> <li>Why Austin? <ul> <li><strong>Written in pure C</strong> Austin is written in pure C code. There are no dependencies on third-party libraries.</li> <li><strong>Just a sampler</strong> - fast: Austin is just a frame stack sampler. It looks into a running Python application at regular intervals of time and dumps whatever frame stack it finds.</li> <li><strong>Simple output, powerful tools</strong> Austin uses the collapsed stack format of FlameGraph that is easy to parse. You can then go and build your own tool to analyse Austin's output.</li> <li>You could even make a <em>player</em> that replays the application execution in slow motion, so that you can see what has happened in temporal order.</li> <li><strong>Small size</strong> Austin compiles to a single binary executable of just a bunch of KB.</li> <li><strong>Easy to maintain</strong> Occasionally, the Python C API changes and Austin will need to be adjusted to new releases. However, given that Austin, like CPython, is written in C, implementing the new changes is rather straight-forward.</li> </ul></li> <li>Creates nice flame graphs</li> <li><p>The Austin TUI is nice! <img src="https://ift.tt/2LfCPQ8" alt="Austin TUI" /></p></li> <li><p>Web Austin is yet another example of how to use Austin to make a profiling tool. It makes use of <a href="https://github.com/spiermar/d3-flame-graph">d3-flame-graph</a> to display a <em>live</em> flame graph in the web browser that refreshes every 3 seconds with newly collected samples. <img src="https://ift.tt/2WEIrZF" alt="" /></p></li> <li><p>Austin output format can be converted easily into the <a href="https://github.com/P403n1x87/austin/blob/master/speedscope.app">Speedscope</a> JSON format. You can find a sample utility along with the TUI and Austin Web.</p></li> </ul> <p><strong>Brian #6:</strong> <a href="https://orbifold.xyz/numbers.html"><strong>Numbers in Python</strong></a></p> <ul> <li>Moshe Zadka</li> <li>A great article on integers, floats, fractions, &amp; decimals </li> <li>Integers <ul> <li>They turn into floats very easily, <code>(4/3)*3</code> → <code>4.0</code>, int → float</li> </ul></li> <li>Floats <ul> <li>don’t behave like the floating point numbers in theory</li> <li>don’t obey mathematical properties <ul> <li>subtraction and addition are not inverses <ul> <li><code>0.1 + 0.2 - 0.2 - 0.1</code> != <code>0.0</code></li> </ul></li> <li>addition is not associative</li> </ul></li> <li>My added comment: Don’t compare floats with ==, use pytest.approx or other approximation techniques.</li> </ul></li> <li>Fractions <ul> <li>Kinda cool that they are there but be very careful about your input</li> <li>Algorithms on fractions can explode in time and to some extent memory.</li> <li>Generally better to use floats</li> </ul></li> <li>Decimals <ul> <li>Good for financial transactions.</li> <li>Weird dependence on a global state variable, the context precision.</li> <li>Safer to use a local context to set the precision locally</li> </ul></li> </ul> <pre><code> &gt;&gt;&gt; with localcontext() as ctx: ... ctx.prec = 10 ... Decimal(1) / Decimal(7) ... Decimal('0.1428571429') </code></pre> <ul> <li>See also <ul> <li><a href="https://docs.python.org/3/library/fractions.html">fractions in std lib</a></li> <li><a href="https://docs.python.org/3/library/decimal.html">decimals in std lib</a></li> <li><a href="https://www.itu.dk/~sestoft/bachelor/IEEE754_article.pdf">What Every Computer Scientist Should Know About Floating-Point Arithmetic</a></li> </ul></li> </ul> <p>Extras:</p> <p>Brian:</p> <ul> <li><a href="https://mail.python.org/archives/list/python-committers@python.org/message/JJWIXYICQHCEFCJCCXVSWTP5O67UVCQC/">python 3.9.0a6</a>, now with the new PEG parser for CPython</li> </ul> <p>Michael:</p> <ul> <li>Keep subscribing over at youtube: <a href="http://pythonbytes.fm/youtube">pythonbytes.fm/youtube</a></li> </ul> <p>Joke:</p> <p><strong><em>Unix is user friendly. It's just very particular about who its friends are. (via</em></strong> <a href="https://pypi.org/project/PyJoke/"><strong><em>PyJoke</em></strong></a><strong><em>)</em></strong></p> <p><strong><em>If you put 1000 monkeys at 1000 computers eventually one will write a Python program. The rest will write PERL.</em></strong> (via <a href="https://twitter.com/JamesAbel">@JamesAbel</a>)</p>

from Planet Python
via read more