Cute python: Implementing "lightweight threading" with the python generator

In the earlier part of "cute Python", David introduces a way to simulate a complete collaboration program with a generator and a simple scheduler. We might be able to extend this scheduler in an intuitive way that allows for extremely lightweight threads for multiple processes. Similar to the stackless Python Micro-threading, the Scareware "lightweight thread" requires little context switching and memory overhead for the OS (and even the user zone) thread. David is here to introduce lightweight threading, a clever solution that solves the problem of using a common solution to deal with a large number of collaborative processes.

The micro-threading domain (at least in Python) has always been a special enhancement part of Stackless python. The topic of stackless and the recent changes it has experienced may well be worth opening a column. But the simple truth is that, under "New Stackless," continuation (continuation) is clearly anachronistic, but the thread is still the reason for the project. This is very complicated ...

At first, let's review some of the content first. So, what is a micro-threading? Micro-Threading is basically a process that can run with very little internal resources--and a process that runs in a single instance of the Python interpreter (in the public memory space, and so on). With the micro-threading, we could run tens of thousands of parallel processes on the current medium-performance PC and switch between contexts hundreds of thousands of times per second. Calls to fork () or standard OS thread calls are simply not up to this level! Even threads in the so-called "lightweight" thread library are several orders of magnitude higher than the micro-threading proposed here.

The meaning of the lightweight threading I've described in this column is a little different from the meaning of OS threads. As far as this is concerned, they are not the same as those provided by Stackless. In many ways, lightweight threads are much simpler than most variants; most of the questions about signals, locks, and the like don't exist. The price of simplicity is that I propose a form of "collaborative multithreading"; I think it's not feasible to add preemption to the standard Python framework (at least in the Stackless Python 2.2-no one knows what __future__ will bring).

Lightweight threading in a sense recalls earlier collaborative multitasking with Windows and MacOS versions (but in a single application). In another sense, however, a lightweight thread is just another way of expressing a stream in a program; all the lightweight threads do (at least in principle) can be done with the "really huge If/elif block" technique (the reckless Programmer's plan for the donkey).

Recall Synergy Program

The previous section of this column presents a mechanism for simulating a synergistic program with a simple generator. The core part of this mechanism is very simple. The scheduler () function wraps a set of generator objects that control the process of delegating the control flow to the appropriate branch. These are not really collaborative programs because they control only the scheduler () function and the branch from that function. But for practical purposes, you can do the same thing with very little extra code. Scheduler () is a code similar to the following:

Listing 1. Scheduler () of Simulation collaborative program

def scheduler(gendct, start):
　　global cargo
　　coroutine = start
　　while 1:
　　　　(coroutine, cargo) = gendct[coroutine].next()

One thing to note about this wrapper is that each builder/collaboration program generates a tuple that contains its intended branch target. The generator/collaboration program basically exits at GOTO target. For convenience, I also let the generator generate a standard cargo container as a way to formalize the data that is transferred between collaborative programs-but you can also transfer data using only global variables or callback Setter/getter functions that have been agreed. Raymond Hettinger has written a Python enhancement initiative (Python enhancement Proposal,pep), which is designed to better encapsulate the data that is being delivered, and perhaps Python will include this initiative in the future.