For an example of how the yield generator in Python uses

Yield is meant to be generated, but in Python it is understood as a generator, and the usefulness of the generator can be iterated, simplifying many of the computational models (and not knowing how to simplify them).
Yield is an expression that has a return value.
When a function contains yield, it is no longer a normal function, but rather a generator. When the function is called, it is not automatically executed, but pauses, see the first example:
Example 1:

>>> def mygenerator ():  ... print ' Start ... '  ... Yield 5 ... >>> mygenerator ()   //called here, and did not print out start ... Indicates that the function that has yield has not been run, that is, pause
 
  
   
  >>> mygenerator (). Next ()  //Call Next () to let the function run. Start ... 5>>> 
 
  

If multiple yields appear in a function then next () will stop at the next yield, see Example 2:
Example 2:

>>> def mygenerator ():  ... print ' Start ... '  ... Yield 5 ... >>> mygenerator ()   //called here, and did not print out start ... Indicates that the function that has yield has not been run, that is, pause
 
  
   
  >>> mygenerator (). Next ()  //Call Next () to let the function run. Start ... 5>>> 
 
  

Why does yield 5 output 5,yield 23 output 23?
We suspect that yield is an expression and there is a return value.
So does this mean that the return value of yield 5 must be 5? In fact, this is not the case, there is a certain relationship with the Send function, this function is essentially similar to next (), the difference is that send is passed the yield expression value in, and next can not pass a specific value, can only pass the none in, so you can think G.next () and G.send (None) are the same. See Example 3:
Example 3:

>>> def fun (): ...  print ' Start ... '  ... m = Yield 5  ... Print M  ... print ' middle ... '  ... D = Yield  ... Print D  ... print ' End ... ' ... >>> m = Fun ()    //Create an Object >>> m.next ()    //will make the function execute to the next yield before start ... 5>>> m.send (' message ')  //Use the Send () Pass value message     //send () pass in the middle ... 12>>> m.next () None      //Visible Next () return value is empty end ... Traceback (most recent): File "
 
  
   
  ", line 1, in 
  
   
    
   stopiteration
  
   
 
  

Use in multiprocess

When Python processes data, memory-heavy data often causes the program to run out of counter-runs or to affect the efficiency of other server programs during operation. This situation tends to change the data collection to traverse through Genertor.

But at the same time, as we know, Generoter seems to only be the process of consumption, so inefficient.
Generator can be consumed by pool.map.

Look at the source code of pool.py.

For I, task in enumerate (TASKSEQ):  ...  Try:   put (Task)  except IOError:   debug (' could not put task on queue ')   break

The reality is to first put all the generator into the queue. Then parallel through the map. This resolves the use of map to parallel.

However, the problem of memory consumption is still not resolved. There are two steps to occupy memory.

1. The first step is the total consumption of generator.
2. The second step is to operate all data in parallel.

Solve the first problem, through some consumption generator to achieve.
Solve the second problem, which can be achieved by IMAP.

The sample code is as follows:

Import multiprocessing as Mpimport Itertoolsimport timedef g (): For El in xrange:  print el  yield elimport osde F f (x): Time.sleep (1) Print str (os.getpid ()) + "" + str (x) return x * xif __name__ = = ' __main__ ': Pool = MP. Pool (processes=4)    # start 4 worker processes go = g () result = [] N = one while True:  g2 = Pool.imap (f, itertools.i Slice (go, N))  if G2: For   i in G2:    result.append (i)    time.sleep (1)  else: Break   print (result)

PS: Use precautions. When produce data, do as little as possible, should be even map is single-threaded to consume data. So try to put the operation in the map. This will make better use of multiple processes to improve efficiency.

