Paul Bissex: How to port 100,000 lines of Python 2 to Python 3

TLDR: Use futurize and its great documentation.

Last summer I led the conversion of a 77KLOC Python 2 web application to Python 3, and that was my guide.

First, we did the Stage 1 conversion, which makes the whole codebase compatible with Python 2 and Python 3. As the docs say, “the goal for this stage is to create most of the diff for the entire porting process, but without introducing any bugs.”

(Note that this doesn’t mean the result works in Python 3; it probably won’t. But syntax and library issues are taken care of, and nothing is broken for Python 2.)

Then we did Stage 2; the end result of that is “Python 3-style code that [also] runs on Python 2 with the help of the appropriate builtins and utilities in future.”

Then came the longest and most challenging stage (which was several weeks of work for me), summarized in a single sentence in the futurize docs:

After running futurize, we recommend first running your tests on Python 3 and making further code changes until they pass on Python 3.

A mere matter of programming.

I had to touch about 25% of our 2400 unit tests to complete that work. Most of the issues were around string handling. A lot of our tests were checking for string (str) values in Django HttpResponse.content — which under Python 3 is a bytestring (bytes). So, under Python 3 a lot of those tests just threw TypeError.

I gave talks on this at Django Boston last fall, and NERD Summit this spring.

Here's the video from the more recent one:

from Planet Python
via read more