How to Learn: Python learning path (5): map reduce learning Preface
Yesterday's blog was deprecated by the Home Page. Although I had a bit of water, I thought it was not worth a lot of effort to view the Help content.
Speaking of map reduce, the first reaction is Hadoop's map reduce function programming.
But the python we want to talk about here has time to write the Hadoop map reduce.
Lamdba Functions
To understand map reduce, you must first understand the Lamdba function. As the name suggests, the Lamdba function is an anonymous function. Many times in the garden, when the competition between c # and java arises, anonymous functions will be displayed as an advantage of c #. In a sense, this is a recognition of anonymous function capabilities. In the latest version of Java, anonymous functions are also planned to be added.
The so-called anonymous function is a function without a name, how to call a function without a name.
Normal Functions and their calls:
- Def f (x ):
- Return 2 * x
- Print f (3)
- # Result
- 6
F is the function name.
F (3) is the call function.
Abnormal functions (anonymous functions) and their calls:
- G = lambda x: x * 2
- Print g (3)
- # Result
- 6
What is G?
It seems to be the function name.
Let's make it more thorough.
- Print (lambda x: x * 2) (3)
- # Result
- 6
Even f and g are completely gone.
What is the use of this item?
Map Functions
For all the functions, let me start with this sentence.
- Help (map)
- # Result
- Map (...)
- Map (function, sequence [, sequence,...])-> list
-
- Return a list of the results of applying the function to the items
- The argument sequence (s). If more than one sequence is given,
- Function is called with an argument list consisting of the corresponding
- Item of each sequence, substituting None for missing values when not all
- Sequences have the same length. If the function is None, return a list
- The items of the sequence (or a list of tuples if more than one sequence ).
After reading this help, you should know where the anonymous function is used!
This function is explained in the previous example:
- Print map (lambda x: x * 2, [1, 2, 4])
- # Result
- [2, 4, 6, 8]
A function parameter is a function. The values in the subsequent sequence are input to this function one by one, and a list is returned.
Reduce Function
International practice:
- Help (reduce)
- # Result
- Reduce (...)
- Reduce (function, sequence [, initial])-> value
-
- Apply a function of two arguments cumulatively to the items of a sequence,
- From left to right, so as to reduce the sequence to a single value.
- For example, reduce (lambda x, y: x + y, [1, 2, 3, 4, 5]) calculates
- (1 + 2) + 3) + 4) + 5). If initial is present, it is placed before the items
- Of the sequence in the calculation, and serves as a default when
- Sequence is empty.
There is also a small example of help this time, so let's run this small example.
- Print reduce (lambda x, y: x + y, [1, 2, 3, 4, 5])
- # Result
- 15
The first parameter function must have two parameters. Otherwise, it cannot be used.
Pass the first and second elements in the sequence as parameters to the function, then pass the return value and the third element to the function, then pass the return value and the fourth element to the parameter, and so on, in fact, the above result is (1 + 2) + 3) + 4) + 5)
Filter Function
Continue international practice:
- Help (filter)
- # Result
- Filter (...)
- Filter (function or None, sequence)-> list, tuple, or string
-
- Return those items of sequence for which function (item) is true. If
- Function is None, return the items that are true. If sequence is a tuple
- Or string, return the same type, else return a list.
Another example is as follows:
- Print filter (lambda x: x % 2 = 1, [1, 2, 4])
- Result
- [1, 3]
People are like their names, filters, and filters out the sequences that meet the requirements.
The first parameter is a function, but it can be none compared to the other parameters.
A function can only return a Boolean value as a condition.
That is to say, all values in the sequence that meet the function judgment conditions are returned.