Generator Implementation of pyston

Generator Implementation of pyston

1. What is the generator of python?

 

Is a function that can be used as an iterator. For example

 

for i in range(10):   print i

 

2. Is it difficult to implement this?

If the user is not allowed to customize the generator function, it is not difficult to implement the built-in functions such as range. Define range as a Class Object, maintain the current value through the internal state, and then define methods such as next. However, it would be terrible to allow users to define such functions. For example, the following code:

 

 

def forfun(): yield hello yield world

def forfun1(n):  i = 0  while i < 10 :     if i < n:       yield i     else :       yield i * 10     i = i + 1

Now it is quite terrible. forfun and forfun1 are compiled into functions, but the functions are stateless (static is not mentioned). It is impossible for each execution to continue after the last return. In pyston, the implementation of generator is similar to that of co-route.

3. ucontex

See http://linux.die.net/man/3/swapcontext

Through the getcontex function, we can obtain a ucontex_t type instance. When modifying this instance through the makecontext function, makecontext needs to provide the called function and parameter list. For example, a piece of code in the page:

 

uctx_func1.uc_stack.ss_sp = func1_stack;uctx_func1.uc_stack.ss_size = sizeof(func1_stack);uctx_func1.uc_link = &uctx_main;makecontext(&uctx_func1, func1, 0);

 

Uctx_func1 is obtained through getconext. First, we need to set the stack used to execute the function and the stack size. uc_link should be the context of the function to be called.

How to skip between each function can also ensure that the Code after the code that has jumped out before is returned to the original function. The key is the swapcontext function, which stores the current context in the ucontext_t pointed to by the first pointer, and then executes the context saved by the second pointer.

 

4. How does generator implement it?

The generator expression or function is an object of the BoxedGenerator type. This Class records the stack of the User-Defined generator function execution, and contains two ucontext_t type variables context and returncontext. When the client continuously calls its next function, the actual execution process is as follows:

Next-> generatorNext-> generatorSend (note that swapcontext (& self-> returnContext, & self-> context) is called );) -> jump to user-defined functions-> execute yield in user-defined code and call the implementation function yield.

-> Yield (in what it does, it calls swapcontext (& self-> context, & self-> returnContext );)

-> At this time, the execution process is returned to generatorSend and the next command of swapcontext is started.

Here, there is still a point missing, that is, when the next call is made for the first time, where the value of context comes from. The key lies in the function of BoxGenerator. The Code is as follows:

 

getcontext(&context);    context.uc_link = 0;    context.uc_stack.ss_sp = stack;    context.uc_stack.ss_size = STACK_SIZE;    makecontext(&context, (void (*)(void))generatorEntry, 1, this);

In generatorEntry, user-defined code is called. Then the ring is linked. The user-defined code contains yield, and yield will return to generatorSend and get the desired result.