12. how to use the decorator in Python

The use of the decorator is an advanced technique in Python Programming. here we have organized 12 steps into the use of the decorator in Python. if you need a decorator, see decorator) is an advanced Python syntax. The decorator can process a function, method, or class. In Python, we have multiple methods to process functions and classes. for example, in the Python closure, we can see the function object as the return result of a function. Compared with other methods, the decorator has simple syntax and high code readability. Therefore, the decorator is widely used in Python projects.

The decorator first appeared in Python 2.5. it was originally used to call objects such as processing functions and methods ). In Python 2.6 and later versions, the decorator is further used for processing classes.

1. Functions
In python, functions are defined by the def keyword, function name, and optional parameter list. Return value using the return keyword. Let's take an example to illustrate how to define and call a simple function:

>>> def foo():...   return 1>>> foo()1

The method body (of course multiple rows are the same) is required. it is represented by indentation. adding double parentheses () after the method name can call the function.

2. Scope
In python, the function creates a new scope. Python developers may say that functions have their own namespaces. This means that when a function encounters a variable, the function will first find it in its namespace. Let's write a simple function to see how the local and global scopes are different:

>>> a_string = "This is a global variable">>> def foo():...   print locals()>>> print globals(){..., 'a_string': 'This is a global variable'}>>> foo() # 2{}

The built-in function globals returns a dictionary containing the names of all the variables known by the python interpreter (for clean and white purposes, I omitted some of the variables created by python ). In #2, I called function foo to print out the content in the local scope of the function. We can see that function foo has its own namespace, although there is nothing in the namespace.

3. variable parsing rules
Of course, this does not mean that we cannot access external global variables in the function. In the scope rules of python, creating a variable will certainly create a variable in the current scope. However, when accessing or modifying a variable, the variable will be first searched for in the current scope, if no matching variable is found, it will be checked in the closed scope in turn. So if we modify the implementation of the function foo so that it can print the variables in the global scope:

>>> a_string = "This is a global variable">>> def foo():...   print a_string # 1>>> foo()This is a global variable

At #1, the python interpreter will try to find the variable a_string. of course, it cannot be found in the local scope of the function, so it will go to the upper scope to find it.
On the other hand, if we assign values to global variables within the function, the results are different from what we think:

>>> a_string = "This is a global variable">>> def foo():...   a_string = "test" # 1...   print locals()>>> foo(){'a_string': 'test'}>>> a_string # 2'This is a global variable'

We can see that global variables can be accessed (if they are variable data types (such as list and dict) or even changed), but they cannot be assigned a value. At #1 in the function, we actually create a local variable to hide the variable with the same name in the global scope. We can draw this conclusion by printing out the content in the namespace. We can also see that the value of the variable a_string printed at #2 has not changed.

4. variable lifecycle
Note that variables not only exist in namespaces, but they all have their own lifecycles. See the following example:

>>> def foo():...   x = 1>>> foo()>>> print x # 1Traceback (most recent call last): ...NameError: name 'x' is not defined

# The error at 1 is not only caused by scope rules (although this is the cause of the NameError thrown), it is also related to the implementation mechanism of function calls in python and many other programming languages. There is no effective syntax for this execution time point in this place so that we can get the value of variable x because it does not exist at all! The namespace of function foo begins and ends with the function call.

5. function parameters
Python allows us to pass parameters to the function. the parameter will become a local variable that exists in the function.

>>> def foo(x):...   print locals()>>> foo(1){'x': 1}

There are many ways to define and pass parameters in Python. for the full version, see the official python documentation. Here is a brief description: The function parameter can be a required location parameter or an optional name, the default parameter.

>>> def foo(x, y=0): # 1...   return x - y>>> foo(3, 1) # 22>>> foo(3) # 33>>> foo() # 4Traceback (most recent call last): ...TypeError: foo() takes at least 1 argument (0 given)>>> foo(y=1, x=3) # 52

In #1, we define the function foo, which has a location parameter x and a naming parameter y. In section #2, we can call a function in a conventional way. although there is a named parameter, the parameter can still be passed to the function through location. When calling a function, we can ignore the named parameter y, as shown in #3. If the name parameter does not receive any value, python automatically uses the declared default value 0. Note that the first position parameter x cannot be omitted. Otherwise, an error occurs as shown in #5.
It is concise and clear, but it may be confusing. Python supports the name parameters used for function calling (I personally think it should be named real parameters ). Let's take a look at the function call at #5. we pass two named real parameters. in this case, because there is a name identifier, the order in which the parameters are passed will not matter.
Of course, the opposite is true: the second parameter of the function is y, but we pass the value to it by position. When the function at #2 calls foo (3, 1), we pass 3 to the first parameter and 1 to the second parameter, although the second parameter is a named parameter.
Sang can't afford it. it seems that it takes a long time to clearly describe such a simple concept: function parameters can have names and locations. This means that the definition and call of a function are slightly different in understanding. We can pass the named parameter (real parameter) to a function that only defines the location parameter, and vice versa! If you do not think it is enough, you can view the official documentation.

6. nested functions
Python allows you to create nested functions. This means that we can define functions in the functions, and the existing scopes and variable lifecycles still apply.

>>> def outer():...   x = 1...   def inner():...     print x # 1...   inner() # 2...>>> outer()1

This example is a little complicated, but it looks okay. Think about what happened in #1: The python interpreter needs to find a local variable named x. after the search fails, it will continue to look for it in the upper scope, the upper-layer scope is defined in another function. For function outer, variable x is a local variable, but as mentioned earlier, function inner can access closed scopes (at least read and modify ). In section #2, we call the function inner. it is very important that inner is only a variable name that complies with the python variable parsing rules, the python interpreter first looks for the matched variable name inner in the outer scope.

7. functions are first-class objects in the python world.
Obviously, functions in python are objects like other things. (Singing loudly here! Functions that contain variables are not so special!

>>> issubclass(int, object) # all objects in Python inherit from a common baseclassTrue>>> def foo():...   pass>>> foo.__class__ # 1
 
  >>> issubclass(foo.__class__, object)True
 

You may never have thought that the function you define has attributes. No way, the function is an object in python. like other things, it may be too academic to describe it. point: in python, the function is just a common value. it is the same as other values. This means that you want to upload a function to another function, or return the function from the function! If you have never thought so, take a look at the following example:

>>> def add(x, y):...   return x + y>>> def sub(x, y):...   return x - y>>> def apply(func, x, y): # 1...   return func(x, y) # 2>>> apply(add, 2, 1) # 33>>> apply(sub, 2, 1)1

This example is not surprising to you. Add and sub are two very common python functions. they accept two values and return a calculated result value. At #1, you can see that the variable preparing to receive a function is just a common variable, just like other variables. In section #2, we call the passed function: "() represents the called operation and the value contained in the called variable. In section #3, you can also see that there is no special syntax for passing functions ." The function name is just a table identifier like other variables.
You may have seen this behavior: "python converts frequently used operations into functions for use as parameters, like passing a function to the key parameter of the built-in sorting function to customize sorting rules. When we regard the function as the return value, we can see the following:

>>> def outer():...   def inner():...     print "Inside inner"...   return inner # 1...>>> foo = outer() #2>>> foo
 
  >>> foo()Inside inner
 

This example may seem more strange. At #1, I return the variable inner that happens to be the function identifier as the return value. There is no special syntax for this: "Return the inner of the function; otherwise, it cannot be called at all. "Do you still remember the variable lifecycle? Every time a function outer is called, the inner of the function is redefined. if it is not returned as a variable, it will no longer exist after each execution.
In #2, we capture the return value-the function inner, and store it in a new variable foo. We can see that when the value of the variable foo is evaluated, it does include the function inner, and we can call it. It may seem a bit strange for the first time, but it is not difficult to understand it, right. Hold on, because the strange turning point is coming soon (Hey, I'm not so cool !)

8. closure
Let's not rush to define what a closure is. let's take a look at a piece of code and simply adjust the previous example:

>>> def outer():...   x = 1...   def inner():...     print x # 1...   return inner>>> foo = outer()>>> foo.func_closure(
 
  ,)
 

In the previous example, we learned that inner is returned by outer as a function and saved in the foo variable, and we can call foo () on it (). But does it run normally? Let's take a look at the scope rules.
Everything works under the python scope rule: "x is a local variable in the function outer. When the function inner prints x at #1, the python interpreter finds the corresponding variable inside the inner. of course, the corresponding variable cannot be found, so it then searches for it in the closed scope, and a match is found.
But from the perspective of the variable life cycle, how can we understand it? Our variable x is a local variable of the function outer, which means that it exists only when the function outer is running. According to our known python running mode, we cannot continue calling the function inner after the function outer returns. when the function inner is called, the variable x no longer exists, A runtime error may occur.
Unexpectedly, the inner of the returned function can work normally. Python supports a feature called function closure. In other words, a function defined in a non-global scope can remember its closed namespace when it is defined. This can be concluded by viewing the func_closure attribute of the function. this attribute contains the values in the closed scope (only the captured values, such as x, if other values are defined in outer, they are not included in the closed scope)
Remember, the inner of the function is redefined every time the function outer is called. Now the value of variable x does not change, so the inner of each returned function will be the same logic. what if we change it a little?

>>> def outer(x):...   def inner():...     print x # 1...   return inner>>> print1 = outer(1)>>> print2 = outer(2)>>> print1()1>>> print2()2

In this example, you can see that the closure-the closed scope remembered by the function-can be used to create a custom function, which is essentially a hard-coded parameter. In fact, we didn't pass parameter 1 or parameter 2 to the function inner. we actually created various custom versions that can print various numbers.
Taking out the closure independently is a very powerful function. in some aspects, you may regard it as an object-oriented technology: outer is like a constructor for inner services, x is like a private variable. There are also many ways to use closures: If you are familiar with the python built-in sorting method parameter key, you may have written a lambda method that sorts a list based on the second element rather than the first element. Now you may also be able to write an itemgetter method, receive an index value to return a perfect function, and pass the parameter key to the sorting function.
However, we will not use closures to do this low thing now (⊙ o ⊙ )...! On the contrary, it makes us feel better again to write a tall ornament!

9. decorator
The decorator is actually a closure that treats a function as a parameter and returns an alternative function. Let's look at it step by step from simplicity to complexity:

>>> def outer(some_func):...   def inner():...     print "before some_func"...     ret = some_func() # 1...     return ret + 1...   return inner>>> def foo():...   return 1>>> decorated = outer(foo) # 2>>> decorated()before some_func2

Take a closer look at the example of the above decorator. We have defined a function outer, which has only one some_func parameter, in which we define a nested function inner. Inner prints a string and calls some_func to obtain its return value at #1. The value of some_func may be different for each outer call, but whatever the value of some_func, we will call it. Finally, inner returns the value of some_func () + 1-we can see the printed string and return value 2 by calling the function stored in the variable decorated at #2, instead of the expected return value 1 from calling function foo.
We can think that the variable decorated is a decoration version and a reinforced version of the function foo. In fact, if we want to write a useful modifier, we may want to replace the original function foo with the decoration version, so we will always get our "enhanced version" foo. To achieve this, you do not need to learn new syntax. simply assign the value to the variable foo:

>>> foo = outer(foo)>>> foo # doctest: +ELLIPSIS
 

Now, any call will not involve the original function foo and get the new decorative version foo. now we will write a useful modifier.
Imagine that we have a library that provides objects similar to coordinates. maybe they are just coordinate pairs of x and y. Unfortunately, these coordinate objects do not support mathematical operators, and we cannot modify the source code. Therefore, we cannot directly add operators. We will perform a series of mathematical operations, so we want to be able to perform a proper addition and subtraction operation on two coordinate objects. these methods can easily be written:

>>> class Coordinate(object):...   def __init__(self, x, y):...     self.x = x...     self.y = y...   def __repr__(self):...     return "Coord: " + str(self.__dict__)>>> def add(a, b):...   return Coordinate(a.x + b.x, a.y + b.y)>>> def sub(a, b):...   return Coordinate(a.x - b.x, a.y - b.y)>>> one = Coordinate(100, 200)>>> two = Coordinate(300, 200)>>> add(one, two)Coord: {'y': 400, 'x': 400}

What should we do if our addition and subtraction functions also require some boundary checks? Otherwise, you can only add or subtract a positive coordinate object. any returned value should also be a positive coordinate. So the expectation is as follows:

>>> one = Coordinate(100, 200)>>> two = Coordinate(300, 200)>>> three = Coordinate(-100, -100)>>> sub(one, two)Coord: {'y': 0, 'x': -200}>>> add(one, three)Coord: {'y': 100, 'x': 0}

We expect that, without changing the coordinate objects one, two, and three, the value of one minus two is {x: 0, y: 0}, and the value of one plus three is {x: 100, y: 200 }. Instead of adding the parameter and return value boundary check logic to each method, let's write a decorator for the boundary check!

>>> def wrapper(func):...   def checker(a, b): # 1...     if a.x < 0 or a.y < 0:...       a = Coordinate(a.x if a.x > 0 else 0, a.y if a.y > 0 else 0)...     if b.x < 0 or b.y < 0:...       b = Coordinate(b.x if b.x > 0 else 0, b.y if b.y > 0 else 0)...     ret = func(a, b)...     if ret.x < 0 or ret.y < 0:...       ret = Coordinate(ret.x if ret.x > 0 else 0, ret.y if ret.y > 0 else 0)...     return ret...   return checker>>> add = wrapper(add)>>> sub = wrapper(sub)>>> sub(one, two)Coord: {'y': 0, 'x': 0}>>> add(one, three)Coord: {'y': 200, 'x': 100}

This decorator can work like the previous decorator example and return a modified function. However, in this example, it can perform some useful checks and formatting on the input parameters and return values of the function, and replace x and y of negative values with 0.
Obviously, in this way, our code becomes more concise: we isolate the logic of the boundary check into separate methods, then we apply it to the places we need to check through the decoration device packaging. Another method is to call the boundary check method at the beginning of the calculation method and before the return value. However, it is undeniable that using the decorator allows us to achieve the purpose of checking the coordinate boundary with the minimum amount of code. In fact, if we are decorating our own defined method, we can make the decoration application more powerful.

10. use the @ identifier to apply the decorator to the function.
Python2.4 supports the use of identifiers @ to apply the decorator to the function. you only need to add the @ and decorator names before the function definition. In the example in the previous section, we replaced the original method with the decorated method:

>>> add = wrapper(add)

This method can wrap any method at any time. However, if we customize a method, we can use @ for decoration:

>>> @wrapper... def add(a, b):...   return Coordinate(a.x + b.x, a.y + b.y)

You need to understand that this method is the same as the previous simple replacement of the original method with the packaging method, python only adds some syntactic sugar to make the decorative behavior more explicit and elegant.

11. * args and ** kwargs
We have completed a useful decorator, but it can only be applied to a specific type of method due to hard encoding. This type of method receives two parameters and passes them to the function captured by the closure. What if we want to implement a decorator that can be applied in any way? For another example, if we want to implement a counter-like decorator that can be applied to any method, we do not need to change any logic of the original method. This means that the decorator can accept functions with any signature as its own decoration method, and can call the decorated method with parameters passed to it.
Coincidentally, Python has a syntax that supports this feature. Read Python Tutorial for more details. When * is used to define a function, it means that the parameters passed through the position will be placed in the variable with the * prefix, so:

>>> def one(*args):...   print args # 1>>> one()()>>> one(1, 2, 3)(1, 2, 3)>>> def two(x, y, *args): # 2...   print x, y, args>>> two('a', 'b', 'c')a b ('c',)

The first function one simply prints out all the passed location parameters. as you can see, in code #1, we only reference the args variable in the function, * args is only used to indicate that location parameters are stored in the args variable when the function is defined. Python allows us to set parameters and capture all other uncaptured location parameters through args, as shown in #2.
* Operators can also be used when a function is called. The meaning is basically the same. When calling a function, a variable with the * sign means that the content in the variable needs to be extracted and used as a location parameter. Let's look at an example:

>>> def add(x, y):...   return x + y>>> lst = [1,2]>>> add(lst[0], lst[1]) # 13>>> add(*lst) # 23

# The code at Part 1 is the same as the code at part #2. in part #2, what python does for us can also be done manually. This is not a bad thing. * args can obtain additional parameters from an iterated list when the call method is large, or, when defining a method, it indicates that this method can accept any location parameter.
The ** mentioned below will be a little more complicated. ** it indicates the parameter Dictionary of the key-value pair, which is similar to the parameter dictionary of *, which is also very simple, right:

>>> def foo(**kwargs):...   print kwargs>>> foo(){}>>> foo(x=1, y=2){'y': 2, 'x': 1}

When defining a function, we can use ** kwargs to indicate that all uncaptured keyword parameters should be stored in the kwargs Dictionary. As mentioned above, argshe kwargs is not part of the python syntax, but when defining a function, using such a variable name is an unwritten convention. Like *, we can also use it when defining or calling a function **.

>>> dct = {'x': 1, 'y': 2}>>> def bar(x, y):...   return x + y>>> bar(**dct)3

12. more general decorations
With this new skill, we can easily write a decorator that records the parameters passed to the function. Here is an example of simply outputting logs to the interface:

>>> def logger(func):...   def inner(*args, **kwargs): #1...     print "Arguments were: %s, %s" % (args, kwargs)...     return func(*args, **kwargs) #2...   return inner

Please note that our function inner can accept any number and type of parameters and pass them to the encapsulated method, which allows us to use this modifier to decorate any method.

>>> @logger... def foo1(x, y=1):...   return x * y>>> @logger... def foo2():...   return 2>>> foo1(5, 4)Arguments were: (5, 4), {}20>>> foo1(1)Arguments were: (1,), {}1>>> foo2()Arguments were: (), {}2

Whatever method we define is called, the corresponding logs will be printed to the output window, as we expected.

Summary

Name binding is the core function of the decorator. This syntax is another embodiment of the Python multi-programming paradigm. Most Python users do not need to define decorator, but may use decorator. In view of the wide use of decorator in Python projects, it is very helpful to understand this syntax.