PyBites: How to Deploy Your Open Source Package to PyPI

In this post I share some useful things I learned deploying an open source package to PyPI.

The app

I built a small PyBitesTips class to consume our Python tips from the command line. The code (project) is here

Speaking of tips, here are some cool things I learned / re-used: - Make a class callable using the __call__ dunder (magic) method. - Use namedtuples and instantiate them with ** keyword args: [Tip(**tip) for tip in resp.json()] - Use paging of results with pydoc.pager. - Break down output creation and printing in different methods (and helpers) which made testing the code easier.

Testing

I put the tests in a separate tests/ subdirectory. This way it's easier to omit them from the package build (see further down).

I also mocked out requests.get, providing a static tips_payload list, and builtins.input to simulate a user searching for various tips.

As mentioned before the abstraction of an individual tip output using _generate_tip_output made it easy to write test_tip_output(pb_tips).

Another thing worth mentioning is the conftest.py I added to the main package folder pytip which has pytest added to sys.path. With that change I could just run pytest in the top project folder (more info).

License

This is as simple as copying an existing one and updating the Copyright banner (MIT example). For more info, check out Choosing a License.

setup.py

This file is your key to making your project pip installable. As per the official documentation, a basic setuptools.setup will do the trick. A few things to highlight as well as extra features I used for my setup.py:

• In classifiers you set the Python versions you support, here I use Python 3.x
• In packages you specify which directories to bundle up, here it's just pytip. Note that I called it pytip instead of src. I discovered that is how it ends up in your virtual environment's site-packages.
• In install_requires you specify any 3rd party dependencies, in this case requests. Note that this makes requirements.txt redundant, because python setup.py install will now pull it in automatically.
• And one of the coolest things I learned: main.py, it allows you to run your package as a console script. So here entry_points > console_scripts makes an alias pytip that points to pytip directory (package) > __main__ module > main function which has some argparse logic to make this a CLI app (read much more about this in Erik's Exploring the Modern Python Command-Line Interface). So when you pip install this package, you can just run pytip, how cool is that, no?!

Wheels and PyPI

This is 80% of the battle. Uploading it to PyPI is actually very easy.

1. Make 2 accounts: PyPI and Test PyPI.
2. Get API tokens for both. Note them down because they only show them once.

3. I highly recommend making this file so you can authenticate to both servers without entering a password ever again:

   $ cat ~/.pypirc
   [distutils]
   index-servers =
       pypi
       testpypi
   
   [pypi]
   username: __token__
   password: ...
   
   [testpypi]
   repository: https://test.pypi.org/legacy/
   username: __token__
   password: ...
   

4. pip install setuptools, wheel and twine.

5. Create your distribution: python setup.py sdist bdist_wheel. This will put a tar.gz and a .whl (wheel) in a dist/ folder (which you should add to .gitignore).

6. Always first upload your package to the Test PyPI to make sure it all works: twine upload --repository testpypi dist/*. This is important because version numbers can only be uploaded once, so it better work before uploading it to the real PyPI.

7. pip install the package from the test index to see if you are happy with the results (here I found out about the "pytip vs src directory name" thing by looking at what got installed in site-packages. And here I tested out if my pytip alias, as defined in console_scripts, actually worked.

8. If all good, upload it to the real PyPI server: twine upload dist/* (no need to specify repository as PyPI is the default).

And that's it, a new open source project on PyPI.

Resources

These resources really helped me going through this process end-to-end:

• Packaging Python Projects
• How to Publish an Open-Source Python Package to PyPI - specially the console entry point / alias trick I picked up from here.
• Configuring a .pypirc File for Easier Python Packaging

"Modern" Python

Setup.py is still the way to go in many cases, but Poetry and the standardized toml file is a strong contender.

Check it out for yourself:

• What the heck is pyproject.toml
• We (Patrick G) used it in our own Karmabot

---

Keep Calm and Code in Python!

-- Bob

from Planet Python
via read more