Step by step, we will explore how to use the simplest code in Python to iterate multiple iterable objects at the same time, for example, reading each row from both files for comparison.
Digress:
Recently, the course requires a deep understanding of the Python language. Because I have been using strong static languages such as Java and C ++ all the time, I feel a lot different when I suddenly use Python.
The intuitive feeling is that in Python, you can always find some exquisite, concise, efficient, and beautiful code, which makes the process of writing code full of surprises for beginners, and gradually like Python. In addition, the official Python manual is very well read, and many problems are clearly described. But in general, I still think the Java method is good :)
A very useful syntax in Python isFor inLoop andIterableThe combination of objects and appropriate use of it can make the code more pleasing to the eye. These 1.1 widely cited examples are text file operations, which are also copied here. Common file operations:
with open("test.txt") as fp:while True:line=fp.readline()if line=="":breakdo_something_with(line)
File operations after the for in loop are introduced:
with open("test.txt") as fp:for line in fp:do_something_with(line)
Obviously, the code is much simpler and clearer! In PythonFileYesIterableObject, so you can directlyFor inIteration. But there is a problem.For inOnly one sequence can be iterated at a time. Here, only one row can be read from one file at a time, and some processing is performed. However, if you want to read one row from two files each time and perform some processing, such as text comparison, can you do this? That is to say, in Python, how can we iterate multiple sequences at the same time, or at least two same-length sequences at the same time? A barely acceptable statement:
with open("a.txt") as fa, open("b.txt") as fb:try:while True:do_something_with(fa.next(), fb.next())except StopIteration:pass
But can it be more concise? It can be assumed that the syntax supports the following syntax:
with open("a.txt") as fa, open("b.txt") as fb:for a,b in fa,fb:do_something_with(a, b)
But this is actually not feasible. here we will get "ValueError: too has values to unpack", becauseInTheFa, fbIt is regarded as a sequence, that is, the following statement is legal:
for fx in fa,fb
In this way, the cycle is two times,FXAssignedFaAndFbBut this is obviously not the desired effect. Python has a method to construct a sequence:
[a,b for a in fa for b in fb]
The constructed sequence is actually equivalent to two nested loops, and the loop structure is equivalent:
for a in fa:for b in fb:
Is there anything in Python that can iterate multiple sequences at the same time? There is a function.Map (function, iterable ,...)It can traverse multiple given sequences at the same time. each time a value is obtained from each sequence to form a tuple object, and then callFunctionAnd input this object. If the length of multiple sequences is different, all other sequences will be filled with None to the maximum sequence length. ExploitationMap ()Function code can be simplified:
with open("a.txt") as fa, open("b.txt") as fb:map(do_something_with, fa, fb)
It looks amazing! HoweverMap ()During executionFunctionAdd the returned value toListMedium,Map ()This will be returned after execution.List. However, this is not required here.List, It seems a little waste, not just to pursue the brief code to give up performance. But is there any other way?
In theory, the function of iterating multiple sequences at the same time is generally not supported at the language level or in the core library. Because such behavior is not basic and general enough, programmers can write a few lines of code to implement it, just as they did before. So let's take a step back and assume that if there is no way to directly iterate multiple sequences at the same time, can we combine the two sequences into a binary sequence and then iterate this sequence? Think of a functionZip ([iterable,...]). As I said earlier,Zip ()Combine multiple sequences into a new oneList, NewListEach element in is a tuples. AndMap ()The difference is that if the length of multiple sequences is different, the length of the final returned sequence is the length of the shortest sequence. ExploitationZip ()The code can be written as follows:
with open("a.txt") as fa, open("b.txt") as fb:for a,b in zip(fa, fb):do_something_with(a, b)
This is what we always want to achieve! However, a careful analysis is still incorrect,Zip ()A newListAndListThe size must be at least the sum of the two files! What we need here is to read one row from each of the two files and then process it. after processing, the space occupied by the two rows will be released, then Continue reading the next row, so this process will not consume too much memory.
This is not the case. What else can I do? I searched the internet for a long time, Baidu, Bing, and stackoverflow several times, but none of them found satisfactory answers. When we were about to despair, we accidentally saw someone on stackoverflow saying that there was something wrong with his code. I just gave a brief glance and didn't even take a closer look at his complete description. I only noticed a word --Izip!At that time, I was thinking, what is this? ThisZip ()What are the differences between functions? Query the Python documentation immediately, and then see an eye-catching word:
"It is similar to zip (), but an iterator instead of list is returned"
WithItertools. izip (* iterable)The code is:
from itertools import izipwith open("a.txt") as fa, open("b.txt") as fb:for a,b in izip(fa, fb):do_something_with(a, b)