Python-map/reduce

Python has built map() -in and reduce() functions.

If you read Google's famous paper "Mapreduce:simplified Data processing on Large Clusters", you can probably understand the concept of map/reduce.

Let's look at map first. The map() function receives two arguments, one is the function, the other is to function the Iterable map incoming function to each element of the sequence sequentially, and returns the result as a new one Iterator .

For example, we have a function f (x) =x2, to function on a list [1, 2, 3, 4, 5, 6, 7, 8, 9] , it can be map() implemented as follows:

  f(x) = x * x                  │                  │  ┌───┬───┬───┬───┼───┬───┬───┬───┐  │   │   │   │   │   │   │   │   │  ▼   ▼   ▼   ▼   ▼   ▼   ▼   ▼   ▼[ 1   2   3   4   5   6   7   8   9 ]  │   │   │   │   │   │   │   │   │  │   │   │   │   │   │   │   │   │  ▼   ▼   ▼   ▼   ▼   ▼   ▼   ▼   ▼[ 1   4   9  16  25  36  49  64  81 ]


Now, we use Python code to implement:

def f (x): ...      return x * x ... >>> r = Map (f, [1, 2, 3, 4, 5, 6, 7, 8, 9])>>> list (R) [1, 4, 9, 16, 25, 36, 49, 64, 81 ]

map()The first parameter passed in is the f function object itself. Since the result r is one Iterator , it is an Iterator inert sequence, so the list() function allows it to calculate the entire sequence and return a list.

You might think that you don't need map() a function, write a loop, or you can calculate the result:

L = [] for in [1, 2, 3, 4, 5, 6, 7, 8, 9]:    l.append (f (n))print(L)

Yes, but, from the loop code above, can you see "putting F (x) in every element of the list and generating a new list"?

So, map() as a higher-order function, in fact it abstracts the arithmetic rules, so we can not only calculate the simple f (x) =x2, but also can calculate any complex function, for example, the list of all the numbers into a string:

>>> list (map (str, [1, 2, 3, 4, 5, 6, 7, 8, 9]))['1','2','3','4','5','6','7','8','9']

Only one line of code is required.

Again look at reduce the usage. To function reduce on a sequence [x1, x2, x3, ...] , the function must receive two parameters, reduce and the result continues and the next element of the sequence is accumulated, the effect is:

Reduce (f, [X1, x2, X3, x4]) = f (f (f (x1, x2), x3), x4)

For example, to sum a sequence, it can be reduce implemented by:

 from Import Reduce def Add (x, y): ...      return x + y ... >>> reduce (add, [1, 3, 5, 7, 9])25

Of course, the sum operation can be directly built into Python functions sum() , no need to use reduce .

But if you want to [1, 3, 5, 7, 9] transform the sequence into integers 13579 , you can put reduce it in handy:

 from Import Reduce def fn (x, y): ...      return x * ten + y ... >>> reduce (FN, [1, 3, 5, 7, 9])13579

This example is not very useful in itself, but if we consider that the string str is also a sequence, with a slight change to the above example, map() we can write the str converted int function:

>>> fromFunctoolsImportReduce>>>deffn (x, y): ...returnX * 10 +y ...>>>defChar2num (s): ... digits= {'0': 0,'1': 1,'2': 2,'3': 3,'4': 4,'5': 5,'6': 6,'7': 7,'8': 8,'9': 9}...     returnDigits[s] ...>>> reduce (FN, map (Char2num,'13579'))13579

str2intthe function that is organized into one is:

 fromFunctoolsImportreducedigits= {'0': 0,'1': 1,'2': 2,'3': 3,'4': 4,'5': 5,'6': 6,'7': 7,'8': 8,'9'8 {}defStr2Int (s):deffn (x, y):returnX * 10 +ydefChar2num (s):returnDigits[s]returnReduce (FN, map (Char2num, s))

You can also use lambda functions to further simplify:

 fromFunctoolsImportreducedigits= {'0': 0,'1': 1,'2': 2,'3': 3,'4': 4,'5': 5,'6': 6,'7': 7,'8': 8,'9'8 {}defChar2num (s):returnDigits[s]defStr2Int (s):returnReduceLambdaX, y:x * + y, map (Char2num, s))

That is, assuming that Python does not provide a int() function, you can write a function that converts the string to an integer by itself, and only requires a few lines of code!

The use of lambda functions is described later.

Python-map/reduce