Understanding of function calls, closures, adorners in Python

Study CS 61A higher-order Functions Chapter when the doubts encountered, read the answer after the enlightened.

All objects in Python

This is probably the most useful word for learning Python. Presumably you already know Python in the list, tuple, dict and other built-in data structures when you execute:

= [123]

, you create a list object and reference it with the alist variable:

Of course you can also define a class yourself:

class House(object):    def__init__(self, area, city):        self= area        self= city    def sell(self, price):        [...]  #other code        return price

Then create an object of a class:

= House(200'Shanghai')

OK, you immediately have a 200 square meter house in Shanghai, it has some properties (area, city) , and some methods (__init__, self) :

A function is the first class of objects

As with list, tuple, dict, and House the object created, when you define a function, the function is also an object:

def func(a, b):    return a+b

In the global domain, a function object is referenced by a function name, which receives two parameters A and B, evaluates both parameters and acts as the return value.

The so-called first class object, which means that the object can be named with an identifier, and the object can be treated as data processing, such as assignment, passed as a parameter to a function, or as return value return, etc.

Therefore, you can refer to this function object with other variable names:

= func

In this way, you can func(1, 2) call the function with a new reference, just like a call:

print func(12)print add(12)  #the same as func(1, 2)

or pass the function object as an argument to another function:

def caller_func(f):    return f(12)if__name__=="__main__":    print caller_func(func)

can see that

• The function object func is passed as caller_func a parameter to the function, and the pass-through procedure is similar to an assignment operation f=func ;
• So the func function object, which is caller_func referenced by the local variable in the function scope f , f actually points to the function func ;
• When executed return f(1, 2) , the equivalent of execution return func(1, 2) ;

So the output is 3.

Function Object vs Function call

Either assigning a function to a new identifier or passing it as a parameter to a new function is all about the function object itself, not the function's invocation .

Use a simpler, but visually, confusing example to illustrate the problem. For example, the following function is defined:

def func():    return"hello,world"

Then perform the assignment two times, respectively:

= func      #将函数对象赋值给ref1= func()       #调用函数，将函数的返回值("hello,world"字符串)赋值给ref2

Many beginners confuse these two assignments by using Python's built-in functions to type see the results of these two assignments:

In [4type(ref1)Out[4]: functionIn [5type(ref2)Out[5str

As you can see, the ref1 function object itself is referenced, and the ref2 return value of the function is referenced. With the built-in callable function, you can further verify that ref1 it is callable and ref2 is not callable:

In [9callable(ref1)Out[9TrueIn [10callable(ref2)Out[10False

The effect of the transfer is similar to that of the reference.

Closure &LEGB Law

The so-called closure, that is, the statement of the constituent function and the execution environment of these statements are packaged together, the resulting object

It sounds a little complicated, or a chestnut to help you understand. Let's say we define the following in the foo.py module:

foo.py

filename = "foo.py"

Def call_func (f):
return f () #如前面介绍的, f refers to a function object and then calls it

In another func.py module, this code is written:

func.py

Import foo #导入foo. py

filename = "func.py"
Def show_filename ():
Return "FileName:%s"% filename

If name = = "main":
Print Foo.call_func (show_filename) #注意: The location where the call actually occurred is in the Foo.call_func function

When we execute func.py with the python func.py command, the output is:
[email protected]: ~$ python func.py
filename:func.py

It is clear that the value of the filename variable used by the Show_filename () function is the one defined in the same environment (func.py module) as it is. Although the filename variable with the same name is also defined in the foo.py module, the location of the actual call to Show_filename is also within the call_func of foo.py.

In the case of nested functions, the mechanism is more obvious: the closure will capture the entire environment required for the inner-layer function to execute:

enclosed.py

Import Foo
def wrapper ():
filename = "enclosed.py"
Def show_filename ():
Return "FileName:%s"% filename
Print Foo.call_func (show_filename) #输出: filename:enclosed.py

In fact, each function object has a __globals__ property that points to the global namespace where the function is defined:

Show_filename inside Wrappershow_filename. Globals

{
'builtins': <module 'builtin' (built-in);, #内建作用域环境
'file': ' enclosed.py ',
' Wrapper ': , #直接外围环境
' package ': None,
' name': ' main',
' foo ': <module ' foo ' from '/home/chiyu/foo.pyc ';, #全局环境
' Doc': None
}

When the code executes to the return "FileName:%s"% filename statement in Show_filename, the parser looks for the filename variable in the following order:
Local-native function (Show_filename), which is assigned in any way, and is not declared by the Global keyword as the variable of the filename;
Enclosing-The local scope of the direct outer space (upper function wrapper), finds the filename variable (if there are multiple layers nested, it is looked up from the inside out to the outermost function);
Global-Universal space (module enclosed.py), the filename variable assigned at the top of the module;
Builtin-Find the filename variable in the predefined variable name in the built-in module (Builtin);
If a matching filename variable is found on any layer, it is no longer looked up to the outer layers. The Nameerror exception is thrown if the required variable is not found until the builtin layer is still located. This is the variable name resolution: the LEGB rule.

Summarize:
The most important use value of closure is: the context of the storage function execution;
Closures are also followed by LEGB rules in their captured execution environment (the context in which the DEF statement block is located) until they find a variable that meets the requirements, or throws an exception.

1. Decorators & Grammar sugars (syntax sugar)
   So what's the relationship between closures and decorators?

The important features of closures are mentioned above: the storage context, which can be cleverly used in the wrapper of existing functions, thus making it more functional for existing functions. And this is the decorator.

As an example, the code is as follows:

Alist = [1, 2, 3, ..., +]-1+2+3+...+100 = 5050

Def lazy_sum ():
return reduce (lambda x, Y:x+y, alist)

We define a function lazy_sum that sums all the elements in the alist and returns them. Alist is assumed to be a list of integers from 1 to 100:
Alist = Range (1, 101)

But for some reason, I don't want to go back to the results immediately, but at some point later, I'm going to output the results by the calls that appear. So I wrap it with a wrapper function:
def wrapper ():
Alist = Range (1, 101)
Def lazy_sum ():
return reduce (lambda x, Y:x+y, alist)
Return lazy_sum

Lazy_sum = wrapper () #wrapper () returns the Lazy_sum function object

If name = = "main":
Lazy_sum () #5050

This is an example of a typical lazy evaluation. We know that in general, local variables are reclaimed by the garbage collector when the function returns, and can no longer be used. But the alist here is not, it is returned with the return of the Lazy_sum function object (this is inaccurate, is actually contained in the Lazy_sum execution environment, through the __globals__), thereby prolonging the life cycle.

When Lazy_sum () is called in the IF statement block, the parser finds the Alist list from the context (this is the local scope of the wrapper function in the enclosing layer), evaluates the result, and returns 5050.

A similar principle becomes useful when you need to dynamically add functionality to a defined function, such as parameter checking:
def add (A, B):
Return a+b

This is a very simple function: Calculate a+b and return, but we know that Python is a dynamic type + strong type of language, you do not guarantee that the user incoming parameters A and B must be two integer, he may have passed an integer and a string type value:
In [2]: Add (1, 2)
OUT[2]: 3

In [3]: Add (1.2, 3.45)
OUT[3]: 4.65

In [4]: Add (5, ' hello ')

TypeError Traceback (most recent)
/home/chiyu/ in ()
----> 1 Add (5, ' hello ')

/home/chiyu/ in Add (A, B)
1 def add (A, B):
----> 2 return a+b

typeerror:unsupported operand type (s) for +: ' int ' and ' str '

As a result, the parser mercilessly throws a TypeError exception.

Dynamic type: Determines the type of the variable during run time, and Python determines the type of a variable when you assign it to him for the first time;
Strongly typed: There is a mandatory type definition, you have an integer, unless the type of the display is converted, it should never be treated as a string (for example, to try to do an integer and a string + operation);

Therefore, in order to use the Add function more gracefully, we need to perform a parameter check on a and b before performing the + operation. This time the adorner is very useful:
Import logging

Logging.basicconfig (level = Logging.info)

def add (A, B):
Return a + b

def checkparams (FN):
Def wrapper (A, B):
If Isinstance (A, (int, float)) and isinstance (b, (int, float)): #检查参数a和b是否都为整型或浮点型
RETURN FN (A, B) #是则调用fn (A, B) returns the result of the calculation

    #否则通过logging记录错误信息，并友好退出    logging.warning("variable 'a' and 'b' cannot be added")       returnreturn wrapper     #fn引用add，被封存在闭包的执行环境中返回

If name = = "main":
#将add函数对象传入, FN points to add
#等号左侧的add, the return value that points to checkparams wrapper
Add = Checkparams (add)
Add (3, ' hello ') #经过类型检查, does not calculate the result, but logs the log and exits

Note the Checkparams function:
First look at the parameter, FN, when we call Checkparams (add), it becomes a local reference to the function object add;
Inside Checkparams, we define a wrapper function, add the function of parameter type checking, and then call FN (A, B), according to LEGB Law, the interpreter will search for several scopes and eventually (enclosing layer) The FN is found in the local scope of the Checkparams function;
Note that the last return wrapper, which creates a closure, the FN variable (a reference to the Add function object) will be sealed in the execution environment of the closure and will not be recycled with the return of the Checkparams;
When the Add = Checkparams (add) is called, add points to the new wrapper object, which adds the function of parameter checking and logging, while still being able to continue calling the original add for the + operation via the sealed FN.

So calling add (3, ' hello ') will not return the result of the calculation, but rather print out the log:
[email protected]: ~$ python func.py
WARNING:root:variable ' A ' and ' B ' cannot be added

Some people think that the Add = Checkparams (add) is too cumbersome, so Python provides a more elegant way of writing, called Syntax Sugar:
@checkParams
def add (A, B):
Return a + b

This is just a writing optimization, and the interpreter will still convert it to add = Checkparams (add) to execute.

1. Regression problems
   def addspam (FN):
   def new (args):
   Print "Spam,spam,spam"
   RETURN fn (args)
   return new

@addspam
def useful (A, B):
Print a2+b2

First look at the second piece of code:
@addspam adorner, equivalent to useful = Addspam (useful). There is a misunderstanding in this topic: the arguments passed to Addspam are useful the function object itself, not one of its invocation results;

Go back to the Addspam function body:
Return new returns a closure in which FN is sealed in the execution environment of the closure without being recycled with the return of the Addspam function;
The FN is now a reference to useful, and when it executes return fn (args), it is actually equivalent to executing the return useful (args);

Finally, attach a reference diagram during the execution of the code, in the hope that it will help you understand:

Understanding of function calls, closures, adorners in Python