Picking a Python Version: A Manifesto

44 Pages·2015·2.329 MB·English

by  David Mertz

Picking a Python
Version: A Manifesto
From __future__ import Python
David Mertz
Picking a Python Version:
A Manifesto
From __future__ import Python
David Mertz
Picking a Python Version: A Manifesto
by David Mertz
Copyright © 2015 O’Reilly Media, Inc. All rights reserved.
Printed in the United States of America.
Published by O’Reilly Media, Inc., 1005 Gravenstein Highway North, Sebastopol, CA
95472.
O’Reilly books may be purchased for educational, business, or sales promotional use.
Online editions are also available for most titles (http://safaribooksonline.com). For
more information, contact our corporate/institutional sales department:
800-998-9938 or [email protected].
Editor: Meghan Blanchette Interior Designer: David Futato
Production Editor: Kristen Brown Cover Designer: Karen Montgomery
Copyeditor: Gillian McGarvey Illustrator: Rebecca Demarest
April 2015: First Edition
Revision History for the First Edition
2015-03-12: First Release
See http://oreilly.com/catalog/errata.csp?isbn=9781491926970 for release details.
The O’Reilly logo is a registered trademark of O’Reilly Media, Inc. Picking a Python
Version: A Manifesto, the cover image, and related trade dress are trademarks of
O’Reilly Media, Inc.
While the publisher and the author have used good faith efforts to ensure that the
information and instructions contained in this work are accurate, the publisher and
the author disclaim all responsibility for errors or omissions, including without limi‐
tation responsibility for damages resulting from the use of or reliance on this work.
Use of the information and instructions contained in this work is at your own risk. If
any code samples or other technology this work contains or describes is subject to
open source licenses or the intellectual property rights of others, it is your responsi‐
bility to ensure that your use thereof complies with such licenses and/or rights.
978-1-491-92697-0
[LSI]
Table of Contents
Python Versions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
The (Small) Break 2
Moving Forward 3
Unix-Like Systems 4
Python 3 Uptake 6
Is It Enough? 9
Porting. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
2to3 12
six.py 15
Python-Future 16
Library Support 18
Advantages of Python 3.x. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
Unicode Handling 23
The Greatly Improved Email and Mailbox Modules 24
The concurrent.futures Module 24
Coroutine Support and yield from 25
The asyncio Module 25
Views and Iterators Everywhere 26
Function Annotations 26
Other Things 27
Other Implementations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
PyPy 29
PyPy-STM 30
Jython 31
iii
IronPython 32
Cython 33
Numba 34
Python Distributions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
Continuum Analytics’ Anaconda 37
Enthought’s Canopy 37
ActiveState’s ActivePython 38
iv | Table of Contents
Python Versions
There are two major versions of the Python programming language:
the Python 2.x series, and the newer Python 3.x series. The Python
3.x series started with the release of Python 3.0 in December 2008;
since that time, Python 2.7 was released and has continued to
receive point version releases, with Python 2.7.9 being the latest ver‐
sion as of this writing. As a footnote, there was, naturally, a Python
1.x series that was highly successful in the 1990s, but that series has
long been out-of-maintenance.
In his 2014 PyCon keynote (Figure 1-1), Guido van Rossum,
Python’s Benevolent Dictator for Life (sometimes affectionately
called the BDFL or GvR) made it clear that the Python 2.x series will
not continue past Python 2.7.x. The end-of-life for Python 2.7.x is
2020, and no new features will be added to the 2.x series of Python.
Within a series, Python places an especially strong philosophical
emphasis on backward compatibility (in comparison to many other
programming languages). It is extremely rare for a series to break
this backward compatibility in later minor versions; in the few cases
where it has occurred, it is only to address large previously undis‐
covered bugs or security issues, and even then great attention is paid
to affecting as little running code as possible. That said, new Python
versions inevitably add important features, either in the language
itself, its built-in functions, or in standard library support modules.
1
Figure 1-1. Guido van Rossum’s PyCon 2014 keynote address (photo
credit: David Lesieur 2014, CC-BY-SA)
Python developers have numerous versions and implementations to
choose from. Many of us still use Python 2.x and CPython—and
that is still often a good choice—but there are many situations in
which making a different choice makes sense.
Python 3.x, and in particular the Python 3.5 version coming out
soon, offer numerous advantages over the Python 2.x series. New
development should typically be done using Python 3.x, and
although the 2020 end-of-life for Python 2.x isn’t all that close, it is
good for companies and projects to start think of porting plans for
existing code.
The (Small) Break
With the introduction of the so-called “Python 3000” version, which
was released after long deliberation as Python 3.0 and subsequent
Python 3.x versions, the designers deliberately allowed small lan‐
guage incompatibilities to be introduced in the new language series.
A full discussion of this is contained in a Python Enhancement Pro‐
posal: PEP 3000 - Python 3000. Even during the Python 2.x to 3.x
series transition, the philosophy of Python insisted on making
2 | Python Versions
changes as incremental as possible. Unlike in the histories of some
other programming language evolutions, Python 3.x was never
meant as a blue-sky project, and avoids the pitfalls of a “second-
system effect.”1 Python 3.x aims to improve a familiar Python 2.x
language rather than to create a new or fundamentally different lan‐
guage.
While there is a significant subset of Python code that is able to be
version-neutral “out of the box,” most existing codebases require a
small degree of rewriting and/or use of compatibility tools and libra‐
ries to move from Python 2.x to Python 3.x. Real-world code using
the most natural idioms in Python 2.x usually does various things
that are not quite compatible with Python 3.x, and the code that is
version-neutral must be written in a careful way to achieve that.
That is to say, if you wrote some module or script 10 years ago, it is
quite unlikely is will “just run” in Python 3.x with no changes.
Moving Forward
The bottom line is that Python 3.x is a better language than Python
2.x, albeit in a large collection of relatively small ways. But as with
any transition—especially one that introduces genuine backward
incompatibilities—it takes time to move users to the latest version.
To a lesser extent, this is true even within a series: there were (are)
certainly some few long-running and stable programs that ran
(probably still run) on Python 2.1—or even on Python 1.5.2—where
simply keeping the platform consistent was easier than testing a
transition. “If it ain’t broke, don’t fix it!”
However, going forward, new projects should be written for Python
3.x, and actively maintained old projects should transition between
versions or plan on doing so at the earliest opportunity. Still, even
after the 2020 end-of-life, executables already running will not sud‐
denly stop doing so (albeit, changes in underlying operatings sys‐
tems or in supporting hardware might require new versions, but
much the same upgrade issue exists for OSes as for programming
languages, and letting old systems “just run” is the same option in
both cases).
One issue to consider when moving to the latest version of Python is
simply what version of the language comes preinstalled with your
1 Fred Brooks, The Mythical Man-Month. Addison-Wesley, 1975.
Moving Forward | 3
operating system, if any. Microsoft Windows® does not ship with any
real built-in developer tools or languages. Of course, Pythons—in
various versions and distributions—are free to download from
python.org and other sites (as are many other programming lan‐
guages and developer tools). Apple OS X® ships with Python 2.x pre‐
installed (in all recent versions; their future plans are confidential
and presumably strategic), so while freely available, installing
Python 3.x takes at least some extra effort. Linux® distributions have
traditionally shipped with Python 2.x installed, but increasingly the
latest versions of these “distros” use Python 3.x for all of their inter‐
nal tooling, ship with Python 3.x on their default media (either
exclusively or with both Python 3.x and Python 2.x included), and
generally encourage the use of Python 3.x. Of course, for many rea‐
sons similar to those discussed above, upgrading existing hardware
systems from older operating systems is itself an effort, and poses
risks of breaking existing programs and workflows.
Unix-Like Systems
PEP 394, entitled The “python” Command on Unix-Like Systems,
specifies the recommended configuration of Python on Unix-like
systems. This includes Linux distros, of course. It equally applies to
BSD-family Unix systems, such as FreeBSD®, OpenBSD™, NetBSD™,
and even to Apple OS X. Of course, while the Python Software
Foundation can make a recommendation via a PEP, no other entity
is bound to follow such recommendations (some do, some don’t).
The basic purpose of this recommendation is to govern the behavior
of files that use the “shebang convention”—that is, files that look at
the first few bytes of a file to see if they are an executable script by
seeing if they look like:
#!/path/to/executable
Or often indirectly as:
#!/usr/bin/env executable
In quick summary, PEP 394 recommends that within an installed
operating system environment:
• python2 will refer to some version of Python 2.x.
• python3 will refer to some version of Python 3.x.
• Currently, python will refer to the same target as python2.
4 | Python Versions
• Eventually, python will refer to the same target as python3.
• Python 2.x-only scripts should either be updated to be source
compatible with Python 3.x or use python2 in the shebang line.
While some currently shipping systems like Apple OS X only ship
with Python 2.x, others like Arch Linux™ ship with python aliased to
python3 already. In (almost) all cases, explicitly specifying python2
or python3 in the shebang line will resolve any ambiguity.
Python 3 on Fedora and Red Hat
Major Linux distributions generally follow the recommendation of
PEP 394, and furthermore, are moving at a consistent pace towards
general internal use of Python 3.x. For example, Fedora’s wiki docu‐
ments this effort:
The main goal is switching to Python 3 as a default, in which state:
• DNF is the default package manager instead of Yum, which
only works with Python 2
• Python 3 is the only Python implementation in the minimal
buildroot
• Python 3 is the only Python implementation on the LiveCD
• Anaconda and all of its dependencies run on Python 3
• cloud-init and all of its dependencies run on Python 3
Python 3 on Ubuntu
Ubuntu is following a similar path vis-à-vis Python versioning as is
Fedora (and Red Hat). Ubuntu’s wiki describes their goals:
For both Ubuntu and Debian, we have ongoing project goals to
make Python 3 the default, preferred Python version in the distros.
This means:
• Python 3 will be the only Python version installed by default.
Python 3 will be the only Python version in any installation
media (i.e. image ISOs)
• Only Python 3 will be allowed on the Ubuntu touch images.
• All upstream libraries that support Python 3 will have their
Python 3 version available in the archive.
Unix-Like Systems | 5