Python ' s hardest Problem--Gil__python

One of the the Long-form articles I ever posted to this blog is a piece about Python ' s Global interpreter Lock (GIL) en Titled "Python ' s hardest Problem". Two weeks ago, it is posted to Hacker News and sat on the front page for a while, driving a lot of traffic to the blog.

In the discussion on Hacker News, some commenters mentioned ' I had neglected to mention the various ways of working AR Ound the GIL. While this information didn ' t fit the purpose's original article, it is nonetheless useful. In this article I'll describe the various ways the Python community has discovered/created to mitigate the effects of the GIL. I Hope this information are helpful to those who want practical the for "to" advice take of advantage in Python.
Batteries included:multiprocessing
For many workloads for which the "GIL is a bottleneck", one need look no further than the Python standard library. The multiprocessing package trades threads for processes, to great effect. The idea are simple:if a single instance of the Python interpreter are constrained by the GIL, one can achieve gains in con Current workloads by through multiple interpreter processes into place of multiple threads. Helpfully, multiprocessing is written with the same interface as the threading package, so code already using threads Doe SN ' t require a massive rewrite to make use of the multiple processes.

How do does it work in practice? One spawns a process in much the same way one creates a thread. The most visible difference between processes and threads are the amount of access to shared data they permit. A Quick example is useful here. Suppose we are writing a RSS feed reader and want to update our feeds with any new items. We Store the contents of our various feeds as a dictionary whose keys are the URL of the RSS feed and whose values are a l Ist of that feed ' s entries. When we "refresh" Our feeds, we simply want to add new items to the the associated feed list.

The

is clearly a parallelizable task. With threads, we would simply give each thread a subset of the dictionary keys (i.e. feed URLs). For each key it receives, a thread would fetch new items in the associated feeds and append them to the feeds ' s item list in Our dictionary. We Needn ' t be concerned with locking the feed dictionary since we know so each thread would be appending to independent l Ists.

With processes, the work is still divided as before. The default behavior for processes, however, are to not share memory and the process that created them. Global variables are an exception to this, but if you ' re using the global variables on a regular basis we have much more impor Tant things to discuss. Our child processes must share data via messaging rather than shared access to the feed dictionary. The usual issues associated with multithreaded code (like data corruption, deadlocks, etc) are no longer a concern. Since no two processes share access to memory, there is no chance of concurrent.

So, that ' s mostly true. As it happens, there are two primary methods of communication available in multiprocessing:queues and pipes. While the \ Queue class is internally synchronized and, thus, thread and process safe, the Pipe class isn't. If more than one thread or process attempts to read from or write to the same end of the same Pipe, corruption of occur. To protect unsafe operations, multiprocessing makes available the same synchronization as primitives.

Minor synchronization issues aside, all of this sounds great. That's, until we realize that sharing the data via messaging requires us to make copies of the everything we like to share. In our example, the parent process sends a portion of the keys in our dictionary (i.e. the feed URL) to each child proces The S. Copying keys are not a expensive operation. Retrieving the results is another matter.

Each child process must send back the contents of the set of item lists that were updated. If we have many feeds and few processes, the resulting lists May is quite large (in terms of memory usage). Since no data is shared between processes, clearly the parent process must copy of the data a child process sends to it. A workflow that includes copying data, possibly the multiple times, is isn't a recipe for a especially quick program.

To work around this limitations, one can make with the two state-sharing methods that multiprocessing makes: Shared memory and server processes. Shared memory comes in the "the" Value and Array classes, and their names are indicative of what ' re they for. Updates to a Value or Array object is immediately visible to the other processes with access to that object. Needless to say, proper using shared primitives as the synchronization important is memory.

Alternately, the Manager class can be used to manage access to shared state by way of proxy objects. The Manager takes the data to be shared and creates proxies of them. Code interacting with these proxy objects are written as if it were interacting with the underlying data itself. All access and modification of the proxy object, however, goes through the Manager.

One advantage of the manager over shared memory are that this manager need to reside on the same physical as the pro Cesses using the proxy objects. Of course, that means, the using a-Manager is slower than shared memory (even when everything was on the same machine).

Now, with the state-sharing methods provided by multiprocessing, we ' ve come full circle. The burden of managing synchronization when using separate processes for concurrency essentially are place back on the Deve Loper. Once shared state, are introduced, the developer is subject to all attendant headaches with associated co De.

But there ' s a silver lining:processes can make progress on multiple threads of execution. Since A parent process doesn ' t share the GIL with their child processes, all processes can execute simultaneously (subject T o The constraints of the hardware and OS).

PyPy
PyPy is often described as "a Python interpreter written in Python". While this ' s a misleading description in a number of ways, suffice it to say this pypy is a alternative implementation of The Python interpreter that achieves (sometimes drastic) performance gains by using a JIT compiler (not unlike the JVM). The PyPy implementation does not (as many mistakenly believe) does away with the GIL. It's still present and functions much of the same as the GIL in the CPython interpreter.

In August of Armin Rigo (a pypy developer and the creator of Pysco), wrote a post on the PyPy blog that generated Quite a bit of discussion. In it, the he outlined a plan to add support for Software transactional Memory (STM) to PyPy. Software transactional Memory (and hardware transactional Memory (HTM)) treats modification of data as a transaction. A transaction is A atomic operation; It either proceeds in it's entirety or is completely rolled back. In PyPy's case, transactions encapsulate modification of Python objects.

It's an idea this has been around for a quite a while, but one of that ' s receiving more attention due to the planned Duction of hardware transactional Memory into the general purpose CPUs (some of Intel ' s new Haswell CPUs have support for TSX Intel ' s extensions for HTM. In the most aggressive form of HTM, there are no need to use synchronization primitives to protect shared data. Each modification are recorded by the CPU; When a transaction finishes, the CPU checks if anyone else made to the changes in memory. If no other modifications were made, the transaction succeeded and proceeds normally. If a modification was detected, the transaction are rolled back and and a "fallback" routine is executed. The fallback routine determines how (and if) the modification should is retried.

This is a potential game-changer for multithreaded programming. As "Python ' s hardest Problem" described, multithreaded programming is difficult due to both the cognitive load it burdens The developer with and the challenge in debugging and proving the correctness of code. If the hardware (or software) magically handled concurrent access to data without requiring anything from the developer, M Ultithreaded programming would is much easier.

But HTM remains quite experimental and hasn ' t yet gained. This is why, back in, Armin Rigo decided that STM was the most promising avenue for creating a "gil-less" pypy. Progress has been slow for the past two (for all years this reasons on any Open Source project is Progress), slow There are signs. In a post earlier this month, Rigo cited a number of factors that would increase the pace of development and included a Nu Mber of ideas for optimizing the implementation.

The project ' s initial, stated goal is to include STM at a performance penalty between 2x and 5x, with the intention of re Ducing (and eventually eliminating) the penalty in subsequent releases. It remains to being seen if the approach taken by Rigo and others are a viable one, but PyPy ' s STM project is perhaps the Pyth On Community ' s best hope of c-based, gil-less Python interpreter.

Alternative Python implementations
While CPython are the official, "reference" interpreter implementation for the Python language, there are a number of alter Nate interpreters written in languages than C. The two most popular are Jython and IronPython. Why are they of interest? Neither has a GIL.

Jython
Jython is a compiled Python interpreter written in Java. It is the successor to the now defunct Jpython project. The Jython project ' s focus, above all else, are compatibility with CPython (tested using a slightly modified version of CPY Thon ' s extensive regression tests).

So how did Jython does away with the GIL? Actually, it wasn ' t a conscious choice by the Jython developers. Because of the JVM ' s built-in garbage collection, there is no need to copy CPython ' s reference-counting implementation. Since There are no longer reference counts that need to is modified on every object, we are free to remove the GIL (which is primarily used for safe manipulation of reference counts to all pyobjects).

That's not to say which, when using Jython, one can ignore modification of the shared state in one ' s own Python code. The GIL doesn ' t protect one from multithreaded Python programs (otherwise there the would is no need for threading to provide Synchronization primitives). Rather, it protects the interpreter itself from corrupting to the bookkeeping data associated with every Python object.

Still, with no GIL, Jython programs can take full advantage to the cores on a machine (our "Holy Grail"). Jython, however, is isn't without its drawbacks.

For starters, it does not support any C extensions. This is a deal-breaker to many people as a ton of popular Python packages make use C extensions. Additionally, development and feature support lag and behind. The beta for Python 2.7 is released in February's this year (and has not seen a release since). Python 2.5 is the officially supported version of Python in Jython. For reference, 2.5 is released September 2006. So it's fair to say this compatibility is a very real problem for Jython.

Lastly, there a number of areas where Jython (by it own admission) is slower than CPython. Any Python standard library modules written in C (and there are a lot to them), Jython implements in Python. These could are rewritten in Java, but the amount of optimization do CPython ' s c-based modules is pretty extensive. Jython is unlikely to approach the speed of CPython modules written into C any time soon.

IronPython
Just as Jython is a compiled Python interpreter written in Java, IronPython are a compiled interpreter written in C #, Makin G it compatible with the rest of the. NET ecosystem. Much like Jython, the GIL are rendered unnecessary due to the. NET DLR garbage Collector. Also like Jython, IronPython benefits from a JIT compiler to achieve speedups in longer running.

While IronPython programs can take full advantage of multicore hardware, the drawbacks of IronPython are largely the same As those of Jython. C extensions are somewhat supported using "ironclad", a commercial tool developed by Resolver Systems. This support is extends to Python 2.6 extensions on (increasingly rare) 32-bit Windows systems, though, and no code has been committed to the project since 2011. I am The It ' s fair to say that C extensions are basically unsupported (especially in the 2.7 version of IronPython).

Again, performance is a mixed bag. While the JIT compiler give IronPython a leg-up at certain areas, there are many areas in which IronPython is considerably Slower than CPython (especially the built-in data structures that are written to highly-optimized C for CPython). Whether or not your code would run faster or slower on IronPython depends heavily on your workload.

Missing the forest for the trees
we discussion of IronPython and Jython has focused on the fact that neither Ementation has a GIL, but that's really of little interest to most developers using either of them. Both projects were created not to merely implement the CPython interpreter in another language, but to allow Python code t o interact with the other parts of each interpreter ' s ecosystem. Calling Java code from Jython is straightforward. To companies and individual developers operating in a java-centric environment, which is a huge win. Likewise, IronPython allows Python code to interact with the rest of the. NET ecosystem.

So and neither is likely to become the reference implementation for Python, that wasn ' t the goal of either of Begin with. It ' s not fair to judge them on their speed or knock their compatibility with C extensions. That is never their goal. Truly, they are both incredible projects that have been wildly successful at accomplishing what they set?? Most "Vanilla" Python developers won ' t use alternate interpreters to increase performance of multithreaded of code. And that ' s just fine.

Third party libraries
There are a number of libraries that extend the capabilities of the CPython interpreter in S Ome way. None, to me knowledge, affect the GIL in any way. Many people are under the assumption that stackless Python has somehow removed the GIL (it hasn ' t). The same goes for [Eventlet] (http://eventlet.net), Greenlet, Twisted, Gevent, and Tornado (all of which I ' ve been mistake Nly told at some point have no GIL). Some of the packages and frameworks help alleviate the pain associated with the GIL, but all are subject to its Limitati Ons.

In Summary
When I originally wrote "Python's Hardest Problem", my goal is to introduce Python novices to the GIL, and explain its functi On, the discuss a bit of its history. "Hardest" in the title is meant to is interpreted as "most technically challenging," not "most important" or "most intere Sting to everyday programmers, "but that's not how a number of people took it. This is surely due to a lack of clarity on the I part, but I hope that this post would help rectify that. There are likely a number of approaches/tools to dealing with the GIL I did not mention. Feel them out of the comments.

My goal for this post are to enumerate a number of ways the community has dealt with the issues the GIL presents for novice Python programmers. If you are feel any of the information above are incorrect or misleading, please let me know on the comments or via email at JE Ff@jeffknupp.com.