A simple understanding of the generator-based state machine in Python

Simple generators have many advantages. In addition to the process of expressing a class of problems in a more natural way, the generator has greatly improved many inefficiencies. In Python, function calls are expensive, and in addition to other factors, it takes a while to resolve the list of function parameters (among other things, the positional and default parameters are analyzed). Initialization of the framework object also takes some steps to establish (according to Tim Peters on Comp.lang.python, there are more than 100 lines of C programming; I haven't checked the Python source code myself yet). In contrast, restoring a generator is fairly labor-saving, the parameters are resolved, and the framework object is "doing nothing" waiting for recovery (with little additional initialization required). Of course, if speed is the most important, you should not use a dynamic language in which bytecode has been compiled, but even if speed is not the primary consideration, it is better to hurry than to slow down.
Memory state Machine

In another article in front of "cute Python," I introduced the StateMachine class, which allows the user to add as many state handlers as a given machine requires. In the model, one or more states are defined as the end state, and only one state is defined as the initial state (start), which is configured by calling the class method. Each handler has some kind of required structure; The handler performs a series of actions, and then, after a while, it returns to the loop in the Statemachine.run () method with a token that indicates the next state to be desired. Similarly, using the cargo variable allows a state to pass some (unhandled) information to the next state.

The typical use of the StateMachine Class I introduced is to use input in a stateful manner. For example, one of my text processing tools (txt2html) reads a few lines of content from a file, and it needs to be handled in a special way, depending on the category to which each line belongs. However, you often need to look at the context provided in the previous lines to determine which category the current row belongs to (and how it should be handled). The implementation of this procedure, built on the StateMachine class, can define a handler that reads a few lines and then processes them in a way similar to a. Soon, a condition is met so that the next batch of content should be handled by the B handler. A control is passed back to the. Run () loop, indicating the switch to the B state-and any additional rows that should be processed by a B before reading an additional number of lines. Finally, a handler passes its control to a state that is specified as an end state, and the processing stops (halt).

For the specific code example in the previous section, I used a simplified application. Instead of processing multiple lines of content, I handle the stream of numbers generated by the iteration function. Each status handler prints only those numbers that are within the desired number range (as well as some messages about the valid state). When a number in a digital stream reaches a different range, a different handler takes over the "processing". For this part, we'll look at another way to implement the same digital streaming with the generator (with some extra tricks and features). However, a more complex example of the generator might be handled more like the input stream mentioned in the previous paragraph. Let's take a look at the previous state machine. The version of the code has been truncated:
Listing 1. statemachine_test.py

From StateMachine import Statemachinedef Ones_counter (val):  print ' ones state:  ', while  1:    if Val <= 0 or Val >=:      newstate = "Out_of_range"; break    Elif <= val <:      newstate = "twenties";   Break    Elif <= val <:      newstate = "TENS";     Break    Else:      print "@%2.1f+"% val,    val = Math_func (val)  print ">>"  return (newstate, VA L) # ... other handlers ... def math_func (n): from  math import sin  return abs (SIN (n)) *31if __name__== "__main__":  m = statemachine ()  m.add_state ("ONES", Ones_counter)  m.add_state ("TENS", Tens_counter)  m.add_ State ("twenties", Twenties_counter)  m.add_state ("Out_of_range", None, end_state=1)  m.set_start ("ONES")  M.run (1)

Readers should take a look at the previous article if they are interested in the imported StateMachine class and how its methods work.

Using the generator

The full version of the

Generator-based state machine is slightly longer than the code sample I would prefer to cover in this column. However, the following code sample is a complete application and does not need to be imported into a separate StateMachine module to provide support. In general, this version is shorter than the one based on the class (we'll see it's something special and very powerful).
Listing 2. stategen_test.py

From __future__ import generatorsimport sysdef Math_gen (n): # iterative function becomes a generator from math import si  n while 1:yield n n = abs (sin (n)) *31# jump targets not state-sensitive, only to simplify Exampledef jump_to (val): If 0 <= val < 10:return ' ONES ' elif <= val < 20:return ' TENS ' elif <= val < 30:return ' TWEN  TIES ' Else:return ' Out_of_range ' def Get_ones (ITER): Global cargo while 1:print "\nones state:" And while Jump_to (cargo) = = ' ONES ': print "@%2.1f"% cargo, cargo = Iter.next () yield (jump_to (cargo), cargo) def get_t ENS (ITER): Global cargo while 1:print "\ntens state:", while jump_to (cargo) = = ' TENS ': print "#%2.1f"% CA rGO, cargo = Iter.next () yield (jump_to (cargo), cargo) def get_twenties (ITER): Global cargo while 1:print "\ n Twenties state: ", while jump_to (cargo) = = ' Twenties ': print" *%2.1f "% cargo, cargo = Iter.next () yield (j Ump_to (cargo), cargo) def Exit (ITER): jump = Raw_input (' \n\n[co-routine for jump?] '). Upper () print "... Jumping to middle of ", jump yield (jump, Iter.next ()) print" \nexiting from Exit () ... "Sys.exit () def scheduler (GENDCT , start): global cargo coroutine = start while 1: (coroutine, cargo) = Gendct[coroutine].next () if __name__ = = "__mai n__ ": Num_stream = Math_gen (1) cargo = Num_stream.next () gendct = {' ONES ': Get_ones (Num_stream), ' TENS ': Get_tens (Num_stream), ' Twenties ': get_twenties (Num_stream), ' Out_of_range ': Exit (Num_stream)} schedule R (GENDCT, jump_to (cargo))

There are a lot of places to study about generator-based state machines. The 1th is superficial to a large extent. We've arranged for stategen_test.py to use only functions, not classes (at least as I mean, the generator has a sense of functional programming rather than object-oriented programming (OOP).) However, if you want, you can easily wrap the same generic model into one or more classes.

The main function in our sample is scheduler (), which is completely generic (but much shorter than the statemachine in the previous pattern). The function Scheduler () requires the builder-iterator object dictionary (the "instantiated" generator) as a parameter. The string name given to each generator can be any name you want-the literal name of the Generator factory function is an obvious choice, but I use the UPPERCASE keyword name in the example. The scheduler () function also accepts "initial state" as an argument, but you might be able to automatically select a default value if you wish.

Each "dispatched" generator follows some simple conventions. Each generator runs for a period of time and then produces a pair of values that contain the desired jump and a "cargo"-just like the previous model. No generators are explicitly marked as "End State". Instead, we allow each generator to choose to produce an error to end Scheduler (). In special cases, if the generator "leaves" the end state or reaches a return state, the generator will produce a stopiteration exception. You can catch this exception (or a different exception) if you want. In our example, we use Sys.exit () to terminate the application and we will encounter this sys.exit () in the exit () generator.

Be aware of two minor problems with the code. The above sample uses a more concise loop generator-iterator instead of using an iterative function to generate our numerical sequence. Instead of returning "last value", the generator emits only one (infinite/indeterminate) stream of digits with each successive call. This is a small but useful generator sample. Furthermore, the "state transitions" are isolated in a separate function. In the actual program, the state transition jumps are more context-sensitive and may have to be determined in the actual generator body. This approach simplifies the sample. Although it may not be very useful, but you can listen to it, we are able to generate a generator function through a function factory to further simplify, but in general, each generator is not similar to other generators enough to make this approach practical.

Collaborative programs and semi-cooperative programs

The attentive reader may have noticed that, in fact, we unwittingly entered a flow control structure that was much more useful than originally indicated. In the sample code, there is more than just a state machine. In fact, the above pattern is a very effective and common system of cooperative programs. Most readers may need some background knowledge here.

A synergistic program is a collection of program functions that allow arbitrary branching into other control contexts and any recovery of streams from points. The subroutines that we are familiar with in most programming languages are a very limited branch of a generic synergistic program. The subroutine enters only one fixed point at the top and exits only once (it cannot be restored). The subroutine also always sends the stream back to its caller. Essentially, each synergistic program represents a callable continuation--although adding a new word does not necessarily clarify its meaning to someone who does not know the word. Randall Hydean's "Cocall Sequence between" illustration in the Art of assembly is a great help in interpreting the collaborative process. A link to this diagram is available in the Resources section. There is also a link to Hyde's comprehensive discussion, which is pretty good.

Regardless of the negative impact, you'll notice that the infamous goto statement in many languages also allows arbitrary branching, but in a less structured context it can cause "macaroni code".

The Python 2.2+ Builder is a big step forward for the collaborative program. This big step is that the generator--and the function/subroutine is different--is recoverable, and can get the value after multiple calls. However, the Python generator is nothing more than a "semi-cooperative program" described by Donald Knuth. The generator is recoverable and can be branched elsewhere-but it can only be branched back to the caller who directly invokes it. To be exact, the generator context (as in any context) can invoke other generators or functions on its own--or even recursively--but each final return must be passed through a linear hierarchy of the return context. The Python Builder does not take into account the common collaborative program usage of "producer" and "Consumer" (you can continue at will from the middle of each other).

Fortunately, it's fairly easy to emulate a well-equipped collaborative program with a Python generator. The simple trick is the scheduler () function, which is very similar to the generator in the sample code above. In fact, the state machine that we put forward is itself a much more common framework pattern of cooperative programs. Adapting to this pattern overcomes the small flaws that still exist in the Python generator (so that careless programmers can also play the full power of the macaroni code).

Stategen in operation

The simplest way to get an accurate idea of what's going on in stategen_test.py is to run it:
Listing 3. Run Stategen (Manual jump Control)

% Python stategen_test.pyones state:    @ 1.0TWENTIES State:  *26.1  *25.3ones State:    @ 4.2TWENTIES state:< c4/>*26.4  *29.5  *28.8tens State:    #15.2  #16.3  #16.0ONES State:    @ 9.5  @ 3.8TENS State:    #18.2  #17.9TWENTIES State:  *24.4tens State:    #19.6TWENTIES State:  *21.4tens State:    # 16.1  #12.3ONES State:    @ 9.2  @ 6.5  @ 5.7TENS state:    #16.7TWENTIES State:  *26.4  * 30.0[co-routine for a jump?] twenties ... Jumping to middle of twentiestwenties State:tens State:    #19.9TWENTIES State:  *26.4  *29.4  *27.5< C29/>*22.7tens state:    #19.9TWENTIES State:  *26.2  *26.8exiting from Exit () ...

This output is basically identical to the output in the previous statemachine_test.py. Each row in the result represents a stream that is used in a particular handler or generator, and the flow context is declared at the beginning of the row. However, whenever another orchestration branch goes inside the generator, the build version resumes execution (within a loop), rather than just calling the handler function again. It is less obvious that all get_* () Cooperative bodies are contained in an infinite loop.

To understand the essential differences in stategen_test.py, see what happens in the exit () generator. The first time the generator-iterator is called, a jump target is collected from the user (a simple case of event-driven branching decisions that can be exploited in real-world applications). However, when you call Exit () again, it is in the context of a later stream of the generator-the exit message is displayed and Sys.exit () is called. The user in the interaction sample can jump directly to "Out_of_range" and exit without going to another "handler" (but it will execute a recursive jump into the same generator).

Conclusion

As I said in the introduction, I expect the state machine version of the collaboration program to run much faster than the previous class (Class-with-callback-handler) version with the callback handler. Recovery Generator-iterators are much more efficient. The particular example is so simple that it is hardly enough to be judged, but I welcome the reader's feedback on the specific results.

But no matter what progress I might make in the "collaborative programming model" in terms of speed, it will eclipse the amazing universal flow control it achieves. Many readers on the Comp.lang.python News group have asked how versatile the new Python builder is. I would like to answer the usability of the framework I have described: "As you want!" "For most Python-related things, it's usually much easier to program some things than to understand them." Try my pattern; I think you'll find it useful.

