Supporting Python 3 (Support Python3)--Migration strategy

Migration policy

There is a high risk of making a backward incompatible version of the software. When people need to rewrite their software or maintain a fragmented version in order to support versions of two languages or frameworks, the risk is that they never make the transition and stay on the old version forever, Or worse, that they the switch to another framework.

For the reason Python versions 2.6 and 2.7 include both several forward compatibility features to enable the write cod E for both Python 2 and Python 3, as-well-as-a-migrating in the form of 2to3, a-program and the-package-that can con Vert cod from python 2 to Python 3. There is techniques and strategies you can use and there is also different ways to use 2to3. Which strategy to use depends very much on what type of software is converting.

Only supporting Python 3

The easiest case is if you are only need to support one version of Python at a time. In these cases you can just convert your code to Python 3 and forget about Python 2. With this strategy, you'll first use the 2to3 tool, and the automatic conversion of the most, the changes and then fix EV Ery problem that remains manually in the Python 3 code. You'll probably also want to go through all of the converted code and clean it up, as the 2to3 conversions could not alway S is the most elegant solution for your case.

Separate branches for Python 2 and Python 3

If you need to continue to support Python 2, the simplest case was having a branch on your source tree with the code for Py Thon 2 and another branch for Python 3. You'll then has the to do every change on the different branches, which was a bit more work, but feasible if the code Doesn ' t change that often.

One problem with this strategy are that your distribution becomes complex, because-now has both distributions and you n Eed to make sure this Python 3 users get the Python 3 version and Python 2 users get the Python 2 version of the Your package. Solutions for this is documented in distributing packages.

Converting to Python 3 with 2to3

In complex cases your can support both Python 2 and Python 3 by maintaining the source code in a Python 2 version and Conve Rting it with 2to3 to Python 3 support. That's means you'll have to run 2to3 each time you had made a code change so you can test it under Python 3 and on the O Ther Hand2to3 deals with many of the differences.

To does this need a script that performs the 2to3 conversion, because doing all the steps manually quickly becomes RE Ally Boring. Since you need to does the conversion every time you had changed something so can run the tests under Python 3, yo U want to run the conversion has been modified as the conversion can rather slow. That means, the conversion script should compare time stamps on the files to see which ones has been modified and con Vert only them, which means the script can is not a trivial shell script.

You can of course write these conversion scripts yourself, but you might not need to. If you is the using Distutils it has support for running 2to3 as a part of the build process. This also solves the distribution problems, as this is the can distribute only Python 2 code and 2TO3 would be run on that Code during install when installed on Python 3. That's the separate packages or even-copies of the code in your package. Distributing Packages also have information on this.

However, the lazy coders approach here would is to use distribute, as it includes some extensions to the 2to3-story.

Using distribute to support the 2TO3 conversion

DISTRIBUTE[1] is a fork of Phillip J. Eby ' s popular setuptools package and provides Python 3 compatibility, as well as ext Ensions simplifying the support for Python 2 and Python 3 from the same source. Basically what distribute have done are to extend the principles of the Distutils Build_py_2to3command and integrated 2to3 I Nto all parts of the packaging stories.

These changes'll be merged back to Setuptools during, but at the time of writing Setuptools doesn ' t support Pytho N 3.

With distribute-can add a few extra parameters in the setup.py file to has 2to3 run the conversion at build time. This means-need to has one version of the source in your version control system and you therefore only need to f IX Bugs once. You also need only one source release, so if only has the to release the software once and there are only one the Load and install for both Python 2 and Python 3.

You still need to run your tests under all versions of Python so you want to support, but distribute includes a test COM Mand that would convert your code WITH2TO3 before running the tests. You can easily the set up your the package. Then testing becomes just running Python setup.py test once for every Python version of you want to support.

The main drawback with this solution is so you can ' t use the earliest versions of 2to3, because they is too buggy. In practice it means your need to having Python 3.1 or later installed on the target machine. This was generally not a problem, as the most platforms, the support Python 3 already use Python 3.1 for.

You can find examples of what to set up your module or package to use distribute for your Python 3 support under Suppor Ting multiple versions of Python with distribute as well as in the standard distribute documentation[2].

Python 2 and Python 3 without conversion

In many cases it's often perfectly feasible to modify the code so it runs under both Python 2 and Python 3 without NE  Eding any conversion, although you has to apply several tricks to avoid the incompatibilities between Python 2 and Python 3.

Python 2.6 and 2.7 has a lot of forward compatibility, making supporting Python 2.6 and Python 3 much easier than support ing python 2.5 and Python 3. Supporting 2.5 or even older versions means you had to employ more tricks. Python 3.3 also re-introduces the U ' literal for strings, which helps with one of the major difficulties in Supoprting Py Thon 3.

Benjamin Petersons Excellent Six module[3] also helps by wrapping much of the incompatibilities, and since the need to SUP Port older Python versions is shrinking, supporting both Python 2 and Python 3 without conversion are becoming the Preferre D method.

There is also cases where you can be use distribute, or don ' t want to. You could need to distribute your code in a format which is not installable with distutils and therefore not distribute. In those cases you can ' t use Distribute's 2to3support and then using 2to3 are more work and not using 2to3 becomes a more a Ttractive Prospect.

Even if you have 2to3 for your project as a whole, you still may end up with have to write some code so it runs on bot H python 2 and Python 3 without conversion. This is useful for bootstrapping scripts and setup scripts or if your code generates cod from strings, for example to Crea Te command line scripts. You can of course has both separate strings depending on the Python version, or even run 2to3 on the string using Lib2to3. However, in these cases it's generally easier to make the generated code snippets run on all Python versions without 2to3 .

My recommendation for the development workflow if you want to support Python 3 without using 2to3 are to run 2to3 on the CO De once and then fix it-until it works on Python 3. Only then introduce Python 2 support into the Python 3 code, using the six where needed. ADD support for Python 2.7 first, and then Python 2.6. Doing it This is the can sometimes result in a very quick and painless process.

There is also a tool called Python-modernize which would do a2to3-type conversion of your code, but it would keep Python 2 c Ompatibility together with the six library. This can is a good start.

More information on the techniques necessary to does this was in the chaptersupporting Python 2 and 3 without 2to3 Conver Sion.

Using 3to2

The 2to3 tool is a flexible enough for your define what changes should was done by writing "fixers". Almost any kind of Python code conversion are imaginable here and 3to2[4] are a set of fixers written by Joe Amenta that Doe s the conversion from Python 3 to Python 2. This enables the write your code for Python 3 and then convert it to Python 2 before release.

However, there is no distribute support for 3to2 and also Python 2.5 or earlier does not include the required Lib2to3 Packag E. Therefore 3to2currently remains only a interesting experiment, although this could change in the future.

Which strategy is?Applications

Unless your code is a reusable package or framework you probably does not need to support older versions of Python, unless s Ome of your customers is stuck on Python 2 while others demand so you support Python 3. In the most cases you can just the drop Python 2 support completely.

Python Modules and Packages

If You is developing some sort of module or package, other Python developers use would probably like to Suppor T both Python 2 and python 3 at the same time. The majority of your users would run Python 2 for some time-to-come, so-want to give-them access to new functional ity, but if you don't support python 3, the users of Python 3 must find another package to fulfill their need.

If the stable from a functional standpoint, it might being perfectly reasonable to has separate branches in yo UR version control system and make bugfixes to both branches separately, but if your package is under active development y ou probably want to support both PYTHON 2 and python 3 at the same time from the same code base. If you want to use the official 2to3 conversion method, or if you want to try to get the code running under both PYTHON&NB Sp;2 and python 3 without a conversion step depends on what your code does and what versions of the Python you need to Su Pport.

There is cases where you won ' t is able to run the same code under Python 2 and Python 3 without a lot of effort because I T relies so much in Python internals that the code becomes too different. There is also cases where the code was so straightforward, running 2TO3 on it hardly changes it. Most code are somewhere in between and the decision are not always easy. A Good idea was to run 2to3 on your code and look at the differences. If 2to3makes A lot of changes in your code and then you could want to use it to convert the code to minimize the amount of work Arounds.

If you can support only Python 2.6 and later then supporting Python 3 Without2to3 conversion are probably the best option. Especially if you aren ' t much affected by the binary/unicode switch, or if you have need to support Python 3.3 or later.

If you is already releasing your package using Distutils or it descendants setuptools and distribute, then using Distrib Ute ' s 2to3 are easy, and that might are the path of least resistance. You can also start this and change the code bit by bit to something, that doesn ' t need converting.

Frameworks

The benefit of using frameworks when developing doesn ' t only come from the framework itself, but also from the plugins and Extensions available to it. It is therefore important to make it easy for all developers using and extending the framework to switch to Python 3, as Y ou otherwise risk being stuck on Python 2 forever.

If your framework is extended by writing Python packages that uses distutils, setuptools or distribute as a packaging Syst Em This means the users of your framework is already in a good position, as they can use distutils or distribute for the 2to3 conversion to support both new and old versions of the framework.

IF extensions is packaged and distributed in some other-a-than with distutils want to consider making your own Set of transition easier, or stopping support for older Python versions, so that your THIRD-PA scripts Rty Package developers don ' t has to Use2to3.

Summary

In general, if you write end-user software, you can just switch to Python 3, starting with a one-time run of 2to3 on your Code. If you write a python package both Python 2 and Python 3 at the same time, and you can drop Python 2.5 want Support, try first-to-support Python 2 and 3 Without2to3 conversion.

If you need to support Python 2.5 or older, using 2to3 is often the best option.

Footnotes

[1]	Http://pypi.python.org/pypi/distribute

[2]	Http://packages.python.org/distribute/python3.html

[3]	Http://pypi.python.org/pypi/six

[4]	Http://pypi.python.org/pypi/3to2

In the lake to smell Zhang Note: the original http://python3porting.com/strategies.html

Supporting Python 3 (Support Python3)--Migration strategy