Python Bytes: #176 How python implements super long integers

<p>Sponsored by DigitalOcean: <a href="http://pythonbytes.fm/digitalocean"><strong>pythonbytes.fm/digitalocean</strong></a></p> <p><strong>Topic #0: Quick chat about COVID 19</strong></p> <p><strong>Brian #1:</strong> <a href="https://snarky.ca/what-the-heck-is-pyproject-toml/"><strong>What the heck is pyproject.toml?</strong></a></p> <ul> <li>Brett Cannon</li> <li><code>pyproject.toml</code> <ul> <li>PEP 517 and 518 define what this file looks like and how to use it to build projects</li> </ul></li> <li>We’re familiar with it being used for flit and poetry based projects.</li> <li>Not so much with setuptools, but it does work with setuptools.</li> <li>You can add configuration for non-build related activities, such as coverage, tox, even though those tools support their own config files.</li> <li>Black is gaining popularity, probably more so than the use of flit. <ul> <li>Black only uses pyproject.toml for configuration (what little config is available. But there is some.)</li> </ul></li> <li>So. Project adds use of black, ends up configuring with with pyproject.toml, but not specifying build steps, No builds are broken. :(</li> <li>Brett has the answers. </li> <li>Add the following to pyproject.toml. Then go read the rest of Brett’s article. It’s good.</li> </ul> <pre><code> [build-system] requires = ["setuptools &gt;= 40.6.0", "wheel"] build-backend = "setuptools.build_meta" </code></pre> <p><strong>Michael #2:</strong> <a href="https://github.com/JackMcKew/awesome-python-bytes"><strong>Awesome Python Bytes Awesome List</strong></a></p> <ul> <li>By Jack McKew</li> <li>Will be adding to this repo whenever I hear about awesome packages (in my opinion), PRs are welcome for anyone else though!</li> <li>Already has 5 PRs accepted</li> <li>Comes with graphics!!! Like all good presentations should.</li> <li>Some fun projects this made me recall: <ul> <li>Great Expectations - for validating, documenting, and profiling, your data</li> <li><code>pandas-vet</code> - a plugin for <code>flake8</code> that provides opinionated linting for pandas code.</li> <li>GeoAlchemy - Using <code>SQLAlchemy</code> with Spatial Databases.</li> <li><code>vue.py</code> - Provides <code>Python</code> bindings for <code>Vue.js</code>. It uses <code>brython</code> to run <code>Python</code> in the browser.</li> </ul></li> <li>Remember we have speedy search for our content over at <a href="https://pythonbytes.fm/search"><strong>pythonbytes.fm/search</strong></a></li> </ul> <p><strong>Brian #3:</strong> <a href="https://packaging.python.org/guides/publishing-package-distribution-releases-using-github-actions-ci-cd-workflows/"><strong>Publishing package distribution releases using GitHub Actions CI/CD workflows</strong></a></p> <ul> <li>PyPA</li> <li>You’ve moved to flit (or not) and started using GitHub actions to build and test whenever you push to GitHub. So awesome. </li> <li>But now, there’s still a manual step to remember to publish to PyPI.</li> <li>And maybe we should be checking publish more often with the Test PyPI server.</li> <li>This article is a step by step walkthrough.</li> <li>It’s a bit dated, 3.7. So I’m trying to walk through all the steps with my <a href="https://github.com/okken/cards">cards project</a> and it will be finished by the time this episode goes live.</li> <li>Stumbling blocks right now: <ul> <li>I’ve left my email blank, no email for author or maintainer in pyproject.toml, because neither flit, nor pip require it. But PyPI still does. grrrr. <ul> <li>Trying to decide between: normal email, setting up a new email for it, using a me+pypi gmail alias, setting up a new email address just for pypi, etc.</li> </ul></li> <li>test pypi fails due to “file already exists”, so, that’s always gonna be the case unless I bump the version, so gonna have to try to figure out a way around that.</li> </ul></li> </ul> <p><strong>Michael #4:</strong> <a href="https://github.com/willmcgugan/rich"><strong>Rich text for terminals</strong></a></p> <ul> <li>Rich is a Python library for rich text and beautiful formatting in the terminal.</li> <li>Add colorful text (up to 16.7 million colors) with styles (bold, italic, underline etc.) to your script or application.</li> <li>Rich can also render pretty tables, progress bars, markdown, syntax highlighted source code, and tracebacks -- out of the box.</li> <li>Centered or justified text</li> <li>Tables, tables!</li> <li>Syntax highlighted code</li> <li>Markdown!</li> <li>Can replace <code>print()</code> and does pretty printing of dictionaries with color.</li> <li>Good Windows support for the <a href="https://www.youtube.com/watch?v=8gw0rXPMMPE&amp;app=desktop">new Windows Terminal</a></li> </ul> <p><strong>Brian #5:</strong> <a href="https://github.com/giampaolo/psutil"><strong>psutil: Cross-platform lib for process and system monitoring in Python</strong></a></p> <ul> <li>“psutil (process and system utilities) is a cross-platform library for retrieving information on <strong>running processes</strong> and <strong>system utilization</strong> (CPU, memory, disks, network, sensors) in Python. It is useful mainly for <strong>system monitoring</strong>, <strong>profiling and limiting process resources</strong> and <strong>management of running processes</strong>. It implements many functionalities offered by classic UNIX command line tools such as <em>ps, top, iotop, lsof, netstat, ifconfig, free</em> and others.”</li> <li>Useful for an incredible amount of information about the system you are running on: <ul> <li>cpu times, stats, load, number of cores</li> <li>memory size and usage</li> <li>disk partitions, usage</li> <li>sensors, including battery</li> <li>users</li> <li>processes and process management <ul> <li>getting ids, names, etc. </li> <li>cpu, memory, connections, files, threads, etc per process</li> <li>signaling processes, like suspend, resume, kill</li> </ul></li> </ul></li> </ul> <p><strong>Michael #6:</strong> <a href="https://arpitbhayani.me/blogs/super-long-integers"><strong>How python implements super long integers</strong></a></p> <ul> <li>by Arpit Bhayani</li> <li>In C, you worry about picking the right data type and qualifiers for your integers; at every step, you need to think if <code>int</code> would suffice or should you go for a <code>long</code> or even higher to a <code>long double</code>.</li> <li>In python, you need not worry about these "trivial" things because python supports integers of arbitrary size.</li> <li><code>2 ** 20000</code> in C is INF where as in Python’s it’s fine, just at 6,021 digit result. But how!?!</li> <li>Integers are represented as:</li> </ul> <pre><code> typedef struct { PyObject ob_base; Py_ssize_t ob_size; /* Number of items in variable part */ } PyVarObject; </code></pre> <ul> <li>Other types that has <code>PyObject_VAR_HEAD</code> are <ul> <li><code>PyBytesObject</code></li> <li><code>PyTupleObject</code></li> <li><code>PyListObject</code></li> </ul></li> </ul> <pre><code> # Python's number: struct _longobject { PyObject ob_base; Py_ssize_t ob_size; /* Number of items in variable part */ digit ob_digit[1]; }; </code></pre> <ul> <li>A "digit" is base 230 hence if you convert 1152921504606846976 into base 230 you get 100</li> <li>Operations on super long integers <ul> <li>Addition: Integers are persisted "digit-wise", this means the addition is as simple as what we learned in the grade school </li> <li>Subtraction: Same</li> <li>Multiplication: In order to keep things efficient implements the <a href="https://en.wikipedia.org/wiki/Karatsuba_algorithm">Karatsuba algorithm</a> that multiplies two n-digit numbers in O(nlog23) elementary steps.</li> </ul></li> <li>Optimization of commonly-used integers: Python preallocates small integers in a range of -5 to 256. This allocation happens during initialization</li> </ul> <p>Extras:</p> <p>Michael:</p> <ul> <li>We're coming to YouTube, probably. :)</li> <li>npm is joining GitHub</li> </ul> <p>Joke:</p> <p><img src="https://ift.tt/3c1EGDS" alt="" /></p>

from Planet Python
via read more