The following is an example of how to use the iterator, generator, and list parsing in Python.

The following is an example of how to use the iterator, generator, and list parsing in Python.

Iterator: Preliminary Exploration

As mentioned in the previous chapter, a for loop can be used on any iteratable object. In fact, this is true for all iteration tools in Python that scan objects from left to right, these iteration tools include for loop, list parsing, in member relationship testing, and map built-in functions.

The concept of "iteratable object" is quite novel in Python. Basically, this is the generalization of the concept of sequence: if the object actually stores the sequence, alternatively, you can iterate the objects that generate a result once in the tool environment, which is regarded as iteratable.

> File iterator
A file of the built-in data type can also be iterated. It has a method named _ next _. Each call will return the next row in the file. When the end of the file is reached, __next _ will cause a built-in StopIteration exception, rather than returning an empty string.

This interface is the so-called iteration protocol in Python: objects with the _ next _ method will move forward to the next result, and StopIteration will be triggered at the end of a series of results. Any such object is considered iteratable. Any such object can also be traversed using a for loop or other iteration tools, because all iteration tools work internally by calling _ next __in each iteration, and capturing StopIteratin exceptions to determine when to exit.

for line in open('script.py'): print(line.upper(),end='')

The above code is an example of file iteration, and this usage is the most efficient file reading method, mainly has three advantages: This is the simplest way to write, run fast, the memory usage is also the best.

The alternative method is:

for line in open('script.py').readlines(): print(line.upper(),end='')

This call method will read the file to the memory at one time. If the file is too large, the memory will be consumed.

> Manual iteration: iter and next
To support manual iteration of code (with less input), Python3.0 also provides a built-in function next, which automatically calls the _ next _ method of an object. Given an object X, calling next (X) is equivalent to X. _ next _ (), but the former is much simpler.

From a technical point of view, iteration protocols are worth noting. When the for loop starts, it is passed to the iter built-in function to obtain an iterator from the iteratable object. The returned object contains the required next method. The steps to call iter are not required for files, because file objects are their own iterators, but they are not necessarily required for some other built-in data types.

List and many other built-in objects are not their own iterators because they support multiple open iterators. For such an object, we must call iter to start iteration:

L = [1, 2, 3] iter (L) is L # return falseL. _ next _ () # The error I = iter (L) I. _ next _ () I. _ next __()

Although the Python iteration tool automatically calls these (iter ,__ next _) functions, we can also use them to manually apply the iteration protocol.

List parsing: Preliminary Exploration

> Basic list Parsing Knowledge

L=[1,2,3,4,5]L = [x+10 for x in L]

List Parsing is written in square brackets because they are eventually a way to build a new list. They start with an arbitrary expression that we make up. This expression uses a loop variable (x + 10) that we make up ). This is followed by what we should now think of as a part of the for Loop header, which declares the loop variable and a iteratable object (for x in L)

To run this expression, Python executes an iteration that traverses L in the interpreter, assigns x to each element in sequence, and collects the results of the expression on the left of each element. The result list we get is the content expressed by list resolution. For each x in L, it contains a new list of x + 10.

In fact, the analytical expression of the list is not necessary, because all the work that can be completed through the for loop, but the analytical expression of the List is faster than the manual for loop statement (usually twice faster ), because their iterations are executed in the interpreter at the speed of C language rather than manual Python code, especially for large data sets, this is a major performance advantage of list resolution.

When we want to execute an operation on each item in a sequence, we can consider using list resolution.

> Extended list parsing syntax
In fact, list resolution can have more advanced applications. As a particularly useful extension, the nested for loop in the expression can have a related if clause to filter out results that are not really tested.

lines = [line.rstrip() for line in open('script.py') if line[0]='p']

This if clause checks each row read from the file to see if the first character of the row is p. if not, the line is omitted from the result list.

In fact, if we want to, list parsing can become more complex-their complete syntax allows any number of for clauses, each of which has an optional if clause.

New iteratable object in Python3.0

A basic change in Pyton3.0 is that it emphasizes iteration more than Python2.x. Except for the iterations related to built-in types such as files and dictionaries, Dictionary Methods keys, values, and items all return iteratable objects in Python3.0. The benefit of returning an iteratable object instead of returning a result list is that it saves memory space.

> Multiple iterators vs. a single iterator
Multiple iterators: maintain multiple iterators at different positions in their results
Single iterator: Only one iterator can be maintained. After traversing the result, they are used up.
Generally, multiple iterators are supported by returning a new object for iter calls. A single iterator generally means that an object returns itself.

> Dictionary view iterator
In Python3.0, the dictionary's keys, values, and items Methods return iterated view objects, which generate one result item at a time, instead of generating all results lists in memory. View items maintain the same physical order as those in the dictionary and reflect the changes made to the underlying dictionary.

Like all iterators, we can always pass a Python3.0 dictionary view to the list built-in function to force a real list to be built. However, this is generally not necessary.

In addition, the Python3.0 dictionary still has its own iterator, which returns continuous keys. Therefore, you do not need to call keys directly in this environment:

For key in D: print (key, end = '')

> List parsing and map
List resolution applies an arbitrary expression to the values in a sequence, collects the results in a new list, and returns the results. In terms of syntax, list Parsing is encapsulated by square brackets (to remind you that they have constructed a list ). Their simple form is to write an expression in square brackets. Python then collects the results of each iteration in the application loop of this expression. For example, if we want to collect the ASCII code of all characters in a string, we can do this:

# Loop method res = [] for x in 'spam': res. append (ord (x) # map function method res = list (map (ord, 'spam') # list resolution res = [ord (x) for x in 'spam']

> Added test and nested loops.
In fact, list Parsing is more common than the above mentioned. We can write an if branch after for to increase the selection logic.

# List parsing [x ** 2 for x in range (10) if x % 2 = 0] # maplist (map (lambda x: x ** 2 ), filter (lambda x: x % 2 = 0), range (10 ))))

The above two lines of code are collected from 0 ~ The sum of the squares of the even numbers in 9 can be seen clearly. After the same function is completed, list parsing statements are much simpler.

In fact, list parsing can be more common. You can write any number of nested for loops in a list parsing process, and each of them has an optional associated if test. The general structure is as follows:

expression for target1 in iterable1 [if comdition1]      for target2 in iterable2 [if condition2] ...      for targetN in iterableN [if conditionN]

When for clauses are nested in list parsing, they work like equivalent nested for loop statements. For example, the following code:

res=[x+y for x in [0,1,2] for y in [100,200,300]]

The same effect as the following lengthy code:

res=[]for x in [0,1,2]: for y in [100,200,30]:  res.append(x+y)

> List parsing and Matrix
A basic method to compile a matrix using Python is to use a nested list structure. For example, the following code defines two 3x3 matrices:

M=[[1,2,3],  [4,5,6],  [7,8,9]]N=[[2,2,2],  [3,3,3],  [4,4,4]]

List Parsing is also a powerful tool for processing such structures, which can automatically scan rows and columns.

Retrieve all elements in the second column:

[row[1] for row in M]   #[2,5,8][M[row][1] for row in (0,1,2)]  #[2,5,8]

Retrieve the elements on the diagonal line:

[M[i][i] for i in range(len(M))] #[1,5,9]

The following code creates a single-layer list that contains the product of matrix elements.

Copy codeThe Code is as follows:

[M [row] [col] * N [row] [col] for row in range (3) for col in range (3)] # [2, 4, 6, 12, 15, 18, 28, 32, 36]

The following code is more complex to construct a nested list with the same values as above:

Copy codeThe Code is as follows:

[[M [row] [col] * N [row] [col] for col in range (3)] for row in range (3)] # [[2, 4, 6], [, 18], [, 36]

The last one above is hard to understand. It is equivalent to the following statement-based code:

res=[]for row in range(3): tmp=[] for col in range(3):  tmp.append(M[row][col]*N[row][col]) res.append(tmp)

> Understand list Parsing
Based on the tests running in the current Python version, map calls are twice faster than the equivalent for loop, while list resolution is usually a little faster than map calls. The speed gap comes from the underlying implementation. map and list parsing run at the C language speed in the interpreter, which is much faster than Python's for loop in PVM.

Reaccess iterator: Generator

Today's Python provides more support for latency-it provides tools to generate results only when needed, instead of immediately generating results. In particular, there are two language structures that delay result creation as much as possible.

Generator function: it is written as a regular def statement, but a result is returned at a time using the yield statement, and the status of each result is suspended and continues.
Generator expressions: generator expressions are similar to list parsing in the previous section. However, they return an object that generates results on demand instead of creating a result list.
Since neither of the above creates a list at a time, they save memory space and allow computing time to be distributed to each result request.

> Generator function: yield VS return
Previously, all the functions we wrote were regular functions that accept input parameters and immediately return a single result. However, you can write a function that can return a value and continue from where it exits. Such functions are called generator functions because they generate a sequence of values over time.

Generally, generator functions are the same as conventional functions, and are actually written using conventional def statements. Then, when they are created, they automatically implement iteration protocols so that they can appear in the iteration context.

Status pending

Unlike the function that returns a value and exits, the generator function automatically suspends the generated value and continues the function execution. Therefore, they are useful for calculating the entire set of values in advance and manually saving and restoring the status in the class. Since the State saved by generator functions during suspension contains their entire local scope, their local variables keep information and make it available when the function recovers.

The main difference between the generator function and the conventional function is that the generator yield has a value instead of a return value. The yield statement suspends the function and sends a value to the caller. However, the function remains in sufficient state to continue from where it leaves. When the function continues, the previous yield return immediate execution. From a function perspective, this allows its code to generate a series of values over time, instead of computing them at a time and sending them back in content such as the list.

Iterative protocol Integration

An iteratable object defines a _ next _ method. It either returns the next item in the iteration or raises a special StopIteration exception to terminate the iteration. The iterator of an object is accepted by the iter built-in function.

If this protocol is supported, Python's for loop and other iteration technologies use this iteration Protocol to facilitate a sequence or value generator. If this protocol is not supported, the return value is the de-duplicated index sequence.

To support this protocol, a function contains a yield Statement, which is specially compiled as a generator. When called, they return an iterator object that supports an automatic creation method named _ next _ to continue execution. The generator function may also have a return statement, which ends the value generation directly at the end of the def statement block.

Generator Function Application

def gensquares(N): for i in range(N):  yield i ** 2

This function generates a value in each loop and then returns it to its caller. When it is tentative, its previous state is saved and the controller is immediately reclaimed after the yield statement. It allows functions to avoid doing all the work temporarily. This is especially important when the result list is large or it takes a lot of time to process each result. The generator will process the Time Distribution of a series of values in loop iteration.

Extended generator function Protocol: send and next add a send Method to the generator function protocol in Python2.5. The send method generates the next element of a series of results, which is like the _ next _ method, but it provides a method for communication between the caller and the generator, this affects its operations.

def gen(): for i in range(10):  X =yield i  print(X)G = gen()next(G)     #0G.send(77)    #77 1G.send(88)    #88 2next(G)     #None 3

The above code is difficult to understand, and the translation in the book is inferior and not understandable. I checked some information on the Internet and understood it with my own understanding. The running process of the above Code should be like this: a function object is generated, assigned to G, and next () is called () the first value of the generator is 0, so the return value is 0. At this time, the function runs to the yield statement. When the yield statement is met, the function is immediately suspended and saved, waiting for the next iteration. The send () method is called later in the program, and 77 is passed to the yield statement. The yield statement assigns the value passed by send () (77 here) to X, then print it out. Then the function continues to run until yield is met again. This is the second time yield is met, so 1 is returned, and the function is suspended again, waiting for the next iteration. Next, send () is called. Similarly, the passed parameter (88 here) is assigned to X as the return value of yield, then printed, and the function continues to run, until yield is met again, this is the third time, SO 2 is output. Finally, the next () function is called again. In fact, the 'next () 'function transmits a None value. Therefore, the result is None and 3.

Note that next () and send (None) are equivalent. Through the send () method, we can communicate with the generator.

> Generator expression: List parsing encountered by the iterator
In the latest version of Python, the concept of iterator and list parsing forms a new feature of this language-generator expressions. In terms of syntax, generator expressions are like list parsing, but they are expanded in parentheses rather than square brackets.

[x ** 2 for x in range(4)]  #List comprehension:build a list(x ** 2 for x in range(4))  #Genterator expression:make an iterable

In terms of the execution process, the generator expression is very different: instead of building results in the memory, a generator object is returned, and this object will support iteration protocol.

Generator expressions can be considered as memory space optimization. They do not need to construct the entire result list at a time, just like list parsing in square brackets. They may run slowly in practice, so they may be the best choice for very large result sets.

> Generator function VS generator expression
Generator functions and generator expressions are all iterators, and therefore only one active iteration is supported. We cannot have multiple iterators located in different locations in the result set.

Python3.0 parsing syntax

We have already paid attention to list parsing and generator in this chapter. But don't forget, there are two types of resolution expressions available in Python3.0: Set parsing and dictionary parsing.

[x*x for x in range(10)]   #List comprehension:build list(x*x for x in range(10))   #Generator expression:produces items{x*x for x in range(10)}   #Set comprehension:new in 3.0{x:x*x for x in range(10)  #Directionary comprehension:new in 3.0

Note that the last two expressions above construct all objects at a time, and they do not produce results as needed.

Summary

List parsing, set parsing, and dictionary parsing all build objects at one time and return them directly.
Generator functions and generator expressions are not built in the memory. They return a generator object, which supports the iteration protocol and generates results based on the caller's needs.
Both set parsing and dictionary parsing support nested related if clauses to filter elements from the results.
Function traps

> Local variables are static.
The variable names defined by Python in a function are local variables by default. They exist in the function scope and only exist when the function is running. Python static checks the local variables of Python. When the def code is compiled, it does not detect the value assignment statement at runtime. The assigned variable name is treated as a local variable in the function, rather than a statement that is used only after the value is assigned.

> Functions without return statements
In Python functions, return (and yield) statements are optional. Technically, all functions return a value. If the return statement is not provided, the function automatically returns the None object.

Articles you may be interested in:

• Python iterator and Generator
• Example of using the responsibility chain mode and iterator mode in the design mode in Python
• Python iterator and Generator
• List generator learning tutorial in Python
• Python generator expression and list Parsing
• Differences between list generators and generators in Python
• How to implement micro-Thread Programming Using Python Generator