Python iterators & Generators

See the code see what yield and can not understand. Sign .....

Iterators:

First Python has built-in container classes: List, dict,tuple .... We call these container. Container can be used for. In.. to traverse.

So, what makes traversal successful?

is our iterator object. This object allows traversal to be performed.

An iterator typically has two methods :

①__ITER__ () ②next ()

Let's first explain for. In ... How did it proceed:

First, the __iter__ is called, and this function is used to return the iterator object. Then the next () method output is called.

1 classiterators (object):2     3     def __init__(self):4Self.value = 15     6     def __iter__(self):7         return Self8     9     defNext (self):TenSelf.value+=1 One         returnSelf.value A    -A=iterators () -  forValueinchA: the     PrintValue

Like this example above, in for...in. Call the __iter__ method first to return a iterator object, and then call the next method of the Iteration object.

1 classiterators (object):2     3     def __init__(self):4Self.value = 15     6     def __iter__(self):7         returnITER ([])8     9     defNext (self):TenSelf.value+=1 One         returnSelf.value

However, if we get rid of the seventh line of code, the output will be different, when the output is a. Because this time the returned iterator object is ITER ([+].

( Ps:iter is a python built-in function that returns a iterator object. ）

So the next () function is called next (), so the output is a one-to-one.

Iterators know this is almost the same.

Generator:

Generator The popular point is that there is a yield function, the yield statement is a bit like return. Simple little chestnuts:

1 def generator (): 2         yield 13         yield 24     5    6 a=Generator ()  7print  a.next ()8print a.next ()

Output 1, 2.

Here, A=generator () is not an execution function, but instead generates a generator object.

The generator is an iterator.

So the generator also has the next () method, or you can use for ... in ...

1 classText8corpus (object):2     """iterate over sentences from the ' Text8 ' corpus, unzipped from Http://mattmahoney.net/dc/text8.zip."""3     def __init__(Self, fname, max_sentence_length=1000):4Self.fname =fname5Self.max_sentence_length =Max_sentence_length6 7     def __iter__(self):8         #The entire corpus is one gigantic line--there be no sentence marks at all9         #So just split the sequence of tokens arbitrarily:1 sentence = tokensTenSentence, rest = [], b"' One With Utils.smart_open (self.fname) as Fin: A              whileTrue: -Text = rest + fin.read (8192)#avoid loading the entire file (=1 line) into RAM -                 ifText = = Rest:#EOF theSentence.extend (Rest.split ())#return the last chunk of words, too (could be Shorter/longer) -                     ifsentence: -                         yieldsentence -                      Break +Last_token = Text.rfind (b' ')#The last token may has been split in ... keep it for the next iteration -Words, rest = (Utils.to_unicode (Text[:last_token]). Split (), Text[last_token:].strip ())ifLast_token >= 0Else([], text) + sentence.extend (words) A                  whileLen (sentence) >=self.max_sentence_length: at                     yieldSentence[:self.max_sentence_length] -Sentence = Sentence[self.max_sentence_length:]

The problem that I met was solved. This is a piece of code in Word2vec that reads characters from a file. In the for...in ... , the __iter__ method is run first, because there is yield in this method, so it can iterate. The input file is an English word, separated by a space. The final result is a list of up to 1000 characters per list.

But that B ' and last_token I still don't understand AH.

Python iterators & Generators