Explore how to iterate multiple Iterable objects at the same time in Python

Off Topic:

Recently, because the course needs to get into the Python language. Because Java, C + +, and other strongly typed static languages have been used all the time, it is now suddenly true that Python does feel a great difference.

The intuitive feeling is that in Python you can always find something that makes the code smart, concise, efficient, and aesthetically pleasing, making the beginner's process of writing the code full of surprises, and gradually becoming fond of python. And the official Python handbook reads very well and many of the questions are described clearly. But overall, I still think Java Dafa is good:)

A very useful syntax in Python is the combination of a in loop with a iterable object, which makes the code more pleasing to use. The 1.1 widely cited examples are text file operations, which are also copied here. Normal file operation:

With open ("Test.txt") as Fp:while true:line=fp.readline () If line== "": Breakdo_something_with (line)

To introduce a for-in loop after a file operation:

With open ("Test.txt") as Fp:for line in Fp:do_something_with (line)

Obviously, the code is a lot more concise and clear all of a sudden! Because the file in Python is a iterable object, it can be directly iterated by for. But there is a problem where the for-in can only iterate one sequence at a time. In this case, you can only read a single line from one file at a time, and then do some processing. But if you want to read a row from each of the two files each time, and then do some processing, such as text comparison, can you do it? In other words, how can python iterate over multiple sequences at the same time, or at least simultaneously iterate over two equal-length sequences? a barely acceptable notation:

With open ("A.txt") as FA, open ("B.txt") as Fb:try:while True:do_something_with (Fa.next (), Fb.next ()) except Stopiteration:pass

But can it be a little more concise? It can be assumed that if the syntax supports the following notation:

With open ("A.txt") as FA, open ("B.txt") as Fb:for, A and B in Fa,fb:do_something_with

But in fact this is not possible, here will get "Valueerror:too many values to unpack", because in the back of the FA,FB is considered a sequence, i.e. the following statement is legal:

For FX in FA,FB

This is a total loop of two times,Fx has been assigned to FA and fb, but this is obviously not the desired effect. one way to construct a sequence in Python is to:

[A, a for a in FA for B in FB]

The resulting sequence is actually equivalent to two loops nested, and the loop structure is equivalent to:

For a in fa:for B in FB:

Is there anything in python that can iterate over multiple sequences at the same time? Think of a function map (functions, iterable, ...)that can traverse a given sequence at the same time, each time a tuple object is taken from each sequence, and then called function and passed into the object. If the length of multiple sequences is different, all other sequences are padded with none to the length of the longest sequence. Use map () The function code can be simplified to:

With open ("A.txt") as FA, open ("B.txt") as Fb:map (Do_something_with, FA, FB)

It looks really good! but map () During execution, each call to the function The returned value is added to a List in which map () This will be returned when the execution is complete . List . However, this listis not needed here, it is a bit wasteful, not just for the sake of the short code to give up performance. But is there any other way?

In theory, the ability to iterate multiple sequences simultaneously is generally not supported at the language level or in the core library. Because this behavior is not basic and generic, the programmer can write a few lines of code to achieve it, just as before. so step back and assume that if there is really no way to iterate over multiple sequences at the same time, can you combine two sequences into a two-tuple sequence and iterate over the sequence? Think of a function zip ([iterable, ...]) . As we just said,zip () can merge multiple sequences into a new list, and each element in the new list is a tuple object. Unlike map () , if the length of multiple sequences is different, the length of the final sequence returned is the length of the shortest sequence. This can be written using the zip () code:

With open ("A.txt") as FA, open ("B.txt") as Fb:for A, a, and in zip (FA, FB):d O_something_with (A, B)

This is the effect that has always wanted to achieve AH! But careful analysis is not correct,zip () generated a new list, and the size of this list is at least two files of the size of the sum! And all that is needed is to read one row from each of the two files, and then do the processing, and after the processing, the space occupied by the two rows can be freed, and then continue to read the next line, so this process is not too memory-intensive.

That's not going to work, is there any way? on the internet for a long time, Baidu, bing, StackOverflow searched several times, have not found a satisfactory answer. Just in despair, inadvertently saw someone on StackOverflow said his piece of code has a problem. Just a glance, not even a full description of his, only notice a word-izip! I was thinking, what was this? What's the difference between this and the zip () function? Immediately query the Python document, and then saw an instant breaking eyeball words:

"Similar to zip (), but returns a iterator instead of a list"

With Itertools.izip (*iterable), all is said, the code is ultimately:

From Itertools import Izipwith open ("A.txt") as FA, open ("B.txt") as Fb:for, A, izip (FA, FB):d O_something_with (A, B)

Explore how to iterate multiple Iterable objects at the same time in Python