Getting started with concurrent programming using the greenlet package in Python, pythongreenlet

Getting started with concurrent programming using the greenlet package in Python, pythongreenlet

1 motivation

The greenlet package is a by-product of Stackless. It calls microthreads "tasklet ". Tasklet runs in pseudo-concurrency and uses channel for synchronous data exchange.

A "greenlet" is a more primitive micro-thread concept, but it is not scheduled or called coroutine. This is useful when you need to control your code. You can construct a micro-thread scheduler by yourself, or use greenlet to implement advanced control flow. For example, you can recreate the constructor. Unlike the Python constructor, our constructor can call functions nested, and nested functions can also have a yield value. (In addition, you do not need a "yield" keyword. For more information, see the example ).

Greenlet is used as a C extension module for unmodified interpreters.

1.1 example

Assume that the system is controlled by the console program and the user enters the command. Assume that the input is a string of characters. Such a system is as follows:

Def process_commands (* args): while True: line = ''while not line. endswith ('\ n'): line + = read_next_char () if line = 'Quit \ N': print "are you sure? "If read_next_char ()! = "Y": continue # ignore the command process_commands (line)

Now suppose you want to port the program to the GUI, and most GUI is event-driven. They will call the callback function each time the user inputs it. In this case, it is difficult to implement the read_next_char () function. We have two incompatible functions:

Def event_keydown (key ):
??

Def read_next_char ():
?? Wait for the call of event_keydown ()

You may be considering thread implementation. Greenlet is another solution with no lock or close issues. You start the process_commands () function, split it into greenlet, and then interact with the key event, as shown in:

Def event_keydown (key): g_processor.switch (key) def read_next_char (): g_self = greenlet. getcurrent () next_char = g_self.parent.switch () # Jump to the greenlet of the previous layer (main) and wait for the next key to return next_charg_processor = greenlet (process_commands) g_processor.switch (* args) gui. mainloop ()

The execution process in this example is: read_next_char () is called, that is, part of g_processor, And it will switch to its parent greenlet, it is assumed that the execution continues in the top-level Main Loop (GUI main loop ). When the GUI calls event_keydown (), it switches to g_processor, which means the execution will jump back to the place where it was suspended, that is, the switch command in the read_next_char () function. Then the key parameter of event_keydown () will be passed to the switch of read_next_char () and return.

Note that read_next_char () will be suspended and its call stack will be well protected during restoration, so it will return at the called place. This allows the program logic to maintain a beautiful Sequential stream. We do not need to rewrite process_commands () to use a state machine.

2. Use

2.1 Introduction

A "greenlet" is a small independent microthread. You can think of it as a stack frame. The bottom of the stack is the initial call, and the top of the stack is the pause position of the current greenlet. You use greenlet to create a bunch of such stack, and then jump between them for execution. Jump is not absolute: One greenlet must select another selected greenlet to jump to. This will cause the previous one to be suspended and the next one to be restored. The jump between the two greenlets is called a switch ).

When you create a greenlet, it gets an initialized empty stack. When you switch to it for the first time, it starts the specified function and switches out of greenlet. When the function at the bottom of the stack ends, the greenlet stack is empty, and greenlet is dead. Greenlet will also die because of an uncaptured exception.

For example:

from py.magic import greenletdef test1():  print 12  gr2.switch()  print 34def test2():  print 56  gr1.switch()  print 78gr1=greenlet(test1)gr2=greenlet(test2)gr1.switch()

The last line jumps to test1 (), It prints 12, then jumps to test2 (), prints 56, then jumps back to test1 (), prints 34, then test1 () ends, gr1 dies. Then the execution will return to the original gr1.switch () call. Note that 78 is not printed.

2.2 parent greenlet

Now let's see where the execution point goes when a greenlet dies. Each greenlet has a parent greenlet. The parent greenlet is created at the beginning of each greenlet (but can be changed at any time ). The parent greenlet is executed at the original location when the greenlet dies. In this way, greenlet is organized into a tree. The top-level code is called greenlet, which is also the root of the tree, does not run in the greenlet created by the user.

In the above example, gr1 and gr2 both use the primary greenlet as the parent greenlet. If any one dies, the execution point will return to the main function.

Uncaptured exceptions will affect the parent greenlet. If test2 () contains a print error (typo), it will generate a NameError and kill gr2. Then the execution point will return to the main function. Traceback will display test2 () instead of test1 (). Remember, the switchover is not a call, but the execution point can be exchanged in parallel between the parallel stack containers. The parent greenlet defines where the stack first came from.

2.3 instance

Py. magic. greenlet is a greenlet type and supports the following operations:

Greenlet (run = None, parent = None)

Create a greenlet object without executing it. Run is the execution callback, while parent is the parent greenlet. The default value is the current greenlet.

Greenlet. getcurrent ()

Returns the current greenlet, that is, who is calling this function.

Greenlet. GreenletExit

This specific exception does not affect the parent greenlet, which is used to kill a greenlet.

The greenlet type can be inherited. A greenlet is executed by calling its run attribute, that is, the one specified during creation. For subclasses, you can define a run () method without having to strictly abide by the run parameter given in the constructor.

2.4 Switch

The switching between greenlet occurs when the switch () method of greenlet is called. This will redirect the execution point to the called place of greenlet's switch. Or, when greenlet dies, it jumps to its parent greenlet. During the switchover, an object or exception is sent to the target greenlet. This can be used as a convenient way to transmit information between two greenlets. For example:

def test1(x,y):  z=gr2.switch(x+y)  print zdef test2(u):  print u  gr1.switch(42)gr1=greenlet(test1)gr2=greenlet(test2)gr1.switch("hello"," world")

This will print "hello world" and "42", which are output in the same order as the previous example. Note that the parameters test1 () and test2 () are not specified during greenlet creation, but are passed when you switch to this for the first time.

Here is the exact call method:

g.switch(obj=None or *args)

Switch to the execution point greenlet g and send the specified object obj. In special cases, if g has not been started, it will be started; in this case, the parameter will be passed and g. run (* args) will be called ).

Dying greenlet

If a greenlet's run () is over, it returns the value to its parent greenlet. If run () is terminated abnormally, the exception will affect the parent greenlet (unless it is a greenlet. GreenletExit exception, in which case the exception will be caught and returned to the parent greenlet ).

In addition to the preceding situation, the target greenlet receives the sent object as the return value of switch. Although switch () does not return immediately, it will return at a certain point in the future, when other greenlet switches back. When this happens, the execution point is restored to switch (), and switch () returns the object sent by the caller. This means that x = g. switch (y) will send the object y to g, then wait for an object that does not know who sent it, and return it to x here.

Note that any attempt to switch to the dead greenlet will switch to the parent greenlet of the dead greenlet, or the parent of the parent, and so on. The final parent is main greenlet and will never die.

2.5 greenlet methods and attributes

G. switch (obj = None or * args)

Switch the execution point to greenlet g, the same as above.

G. run

Call the executable g and start it. After g is started, this attribute no longer exists.

G. parent

The parent of greenlet. This is writable, but does not allow the creation of cyclic parent relationships.

G. gr_frame

The current top frame, or None.

G. dead

Determine if it is dead

Bool (g)

If g is active, True is returned. If g is not started or ended, False is returned.

G. throw ([typ, [val, [tb])

Switch the execution point to greenlet g, but immediately throw the specified exception to g. If no parameter is provided, the default exception is greenlet. GreenletExit. According to the exception spreading rules, as described above. Note that calling this method is equivalent to the following:

  def raiser():    raise typ,val,tb  g_raiser=greenlet(raiser,parent=g)  g_raiser.switch()

2.6 Greenlet and Python threads

Greenlet can be used with Python threads. In this case, each thread contains an independent main greenlet and has its own greenlet tree. Greenlet cannot be switched between different threads.

2.7 garbage collection of greenlet activities

If there is no reference to the greenlet object (including other greenlet parents), there is still no way to switch back to greenlet. In this case, a GreenletExit exception is generated to the greenlet. This is the only situation where greenlet receives an asynchronous exception. A try .. finally should be provided to clean up resources in greenlet. This function also allows an infinite loop programming style in greenlet. In this way, the loop can be automatically interrupted when the last reference disappears.

If you do not want greenlet to die or place the reference elsewhere, you only need to capture and ignore the GreenletExit exception.

Greenlet does not participate in garbage collection; cyclic reference data of greenlet frames will be detected. Passing a reference to another loop greenlet will cause memory leakage.