The Python decorator, closure, and functools tutorials are described in detail.

The Python decorator, closure, and functools tutorials are described in detail.

Decorators)

The decorator is a design pattern. If a class wants to add some functions of other classes without inheriting or directly modifying the source code, you can use the decorator pattern. In simple terms, the decorator in Python refers to some functions or other callable objects. It uses functions or classes as optional input parameters and then returns functions or classes. The new features added in Python2.6 can be used to implement the decoration design pattern.

By the way, if you are not clear about the concept of Closure in Python, please refer to the appendix after the end of this Article. If there is no concept of Closure, it is difficult to properly understand the decorator in Python.

In Python, the decorator is used as a function or class that uses the @ syntax sugar rhetoric. Now let's use a simple decorator example to demonstrate how to create a function call logger. In this example, the time format of the decorator is used as the input parameter to print the Time of the function call when calling the function decorated by the decorator. This decorator is useful when you need to manually compare the efficiency of two different algorithms or implementations.
 

def logged(time_format):  def decorator(func):   def decorated_func(*args, **kwargs):     print "- Running '%s' on %s " % (                     func.__name__,                     time.strftime(time_format)               )     start_time = time.time()     result = func(*args, **kwargs)     end_time = time.time()     print "- Finished '%s', execution time = %0.3fs " % (                     func.__name__,                     end_time - start_time               )      return result   decorated_func.__name__ = func.__name__   return decorated_func return decorator

Let's look at an example. Here, the add1 and add2 functions are modified by logged. The following is an output example. Note that the time format parameters are stored in the decorated_func function returned ). This is why it is important to understand the closure. Also, note how the returned function name is replaced with the original function name, in case it is used again to prevent confusion. Python does not do this by default.

@logged("%b %d %Y - %H:%M:%S")def add1(x, y):  time.sleep(1)  return x + y @logged("%b %d %Y - %H:%M:%S")def add2(x, y):  time.sleep(2)  return x + y print add1(1, 2)print add2(1, 2) # Output:- Running 'add1' on Jul 24 2013 - 13:40:47- Finished 'add1', execution time = 1.001s3- Running 'add2' on Jul 24 2013 - 13:40:48- Finished 'add2', execution time = 2.001s3

If you are careful enough, you may notice that we have special processing for the return function name, but this is not true for other injection _ doc _ or _ module. Therefore, if the add function has a doc string in this example, it will be discarded. So what should we do? Of course, we can treat all fields like _ name _, but if we do this in each decorator, it is not too tedious. This is why the functools module provides a decorator named wraps to handle this situation. It may be confused when you understand the decorator. However, when you think of the decorator as a receiving letter number as an input parameter and return a function, it is easy to understand. In the next example, we will use the wraps modifier instead of manually processing the _ name _ or other attributes.

The next example will be a bit complicated. Our task is to cache the returned results of a function call for a period of time, and the input parameter determines the cache time. The input parameters passed to the function must be Hash objects, because we use the tuple that contains the call input parameters as the first parameter, and the second parameter as a frozenset object, it contains the key word kwargs and serves as the cache key. Each function has a unique cache dictionary stored in the closure of the function.

Set and frozenset are two built-in collections of Python. The former is a mutable object, and its elements can be changed using add () or remove, the latter is an imutable and hashable object. After being created, the element cannot be changed. It can be used as a dictionary key or an element of another set.

Import timefrom functools import wraps def cached (timeout, logged = False): "" Decorator to cache the result of a function call. cache expires after timeout seconds. "def decorator (func): if logged: print" -- Initializing cache for ", func. _ name _ cache ={}@ wraps (func) def decorated_function (* args, ** kwargs): if logged: print "-- Called function", func. _ name _ key = (args, frozenset (kwargs. item S () result = None if key in cache: if logged: print "-- Cache hit for", func. _ name __, key (cache_hit, expiry) = cache [key] if time. time ()-expiry <timeout: result = cache_hit elif logged: print "-- Cache expired for", func. _ name __, key elif logged: print "-- Cache miss for", func. _ name __, key # No cache hit, or expired if result is None: result = func (* args, ** kwargs) cache [key] = (result, Time. time () return result return decorated_function return decorator to check its usage. We use the decorator to decorate a very basic Fibonacci number generator. This cache decorator will use the memo mode (Memoize Pattern) for the code ). Note how the closure of the fib function stores the cache dictionary, a reference pointing to the original fib function, the value of the logged parameter, and the final value of the timeout parameter. Dump_closure will be defined at the end of the document. >>> @ Cached (10, True )... def fib (n ):... "Returns the N' th Maid number. """... if n = 0 or n = 1 :... return 1... return fib (n-1) + fib (n-2 )... -- Initializing cache for fib >>> dump_closure (fib) 1. dumping function closure for fib: -- cell 0 ={} -- cell 1 = -- cell 2 = True -- cell 3 = 10 >>>>>> print "Testing-F (4) = % d "% fib (4) -- Called function fib -- Cache miss for fib (4,), frozenset ([]) -- Called function fib -- Cache miss for fib (3,), frozenset ([]) -- Called function fib -- Cache miss for fib (2 ,), frozenset ([]) -- Called function fib -- Cache miss for fib (1,), frozenset ([]) -- Called function fib -- Cache miss for fib (0,), frozenset ([]) -- Called function fib -- Cache hit for fib (1 ,), frozenset ([]) -- Called function fib -- Cache hit for fib (2,), frozenset ([]) Testing-F (4) = 5 Class Decorators

In the previous section, we read some function decorators and some tips for using them. Next let's take a look at the class decorators. The class decorator uses a class as the input parameter (a type object in Python) and returns a modified class.

The first example is a simple mathematical problem. Given an Ordered Set P, we define Pd as the reverse set P (x, y) of P <-> Pd (x, y ), that is to say, the element order of the two ordered sets is the opposite. How can this be achieved in Python? Suppose a class defines _ lt _ and _ le _ or other methods to achieve order. Then we can replace these methods by writing a class decorator.
 

def make_dual(relation):  @wraps(relation, ['__name__', '__doc__'])  def dual(x, y):    return relation(y, x)  return dual def dual_ordering(cls):  """Class decorator that reverses all the orderings"""  for func in ['__lt__', '__gt__', '__ge__', '__le__']:    if hasattr(cls, func):      setattr(cls, func, make_dual(getattr(cls, func)))  return cls

The following example uses the decorator to create a new class named rstr and uses the opposite lexicographic Order.

@dual_orderingclass rstr(str):  pass x = rstr("1")y = rstr("2") print x < yprint x <= yprint x > yprint x >= y # Output:FalseFalseTrueTrue

Let's look at a more complex example. Suppose we want the aforementioned logged decorator to be used in all methods of a class. One solution is to add a modifier to each class method. Another solution is to write a class decorator to automatically complete the work. Before doing this, I will take out the logged in the keystore for some minor improvements. First, it uses the wraps modifier provided by functools to complete the work of Fixed _ name. Second, a _ logged_decorator attribute is introduced (A boolean variable set to True) to indicate whether the method has been decorated by the decorator, this class may be inherited, And the subclass may continue to use the decorator. Finally, the name_prefix parameter is added to set the printed log information.
 

def logged(time_format, name_prefix=""):  def decorator(func):    if hasattr(func, '_logged_decorator') and func._logged_decorator:      return func     @wraps(func)    def decorated_func(*args, **kwargs):      start_time = time.time()      print "- Running '%s' on %s " % (                      name_prefix + func.__name__,                      time.strftime(time_format)                 )      result = func(*args, **kwargs)      end_time = time.time()      print "- Finished '%s', execution time = %0.3fs " % (                      name_prefix + func.__name__,                      end_time - start_time                 )       return result    decorated_func._logged_decorator = True    return decorated_func  return decorator

Okay. Let's start writing the class decorator:
 

def log_method_calls(time_format):  def decorator(cls):    for o in dir(cls):      if o.startswith('__'):        continue      a = getattr(cls, o)      if hasattr(a, '__call__'):        decorated_a = logged(time_format, cls.__name__ + ".")(a)        setattr(cls, o, decorated_a)    return cls  return decorator

The following describes how to use the method. Pay attention to how the inherited or overwritten method is handled.

@log_method_calls("%b %d %Y - %H:%M:%S")class A(object):  def test1(self):    print "test1" @log_method_calls("%b %d %Y - %H:%M:%S")class B(A):  def test1(self):    super(B, self).test1()    print "child test1"   def test2(self):    print "test2" b = B()b.test1()b.test2() # Output:- Running 'B.test1' on Jul 24 2013 - 14:15:03- Running 'A.test1' on Jul 24 2013 - 14:15:03test1- Finished 'A.test1', execution time = 0.000schild test1- Finished 'B.test1', execution time = 1.001s- Running 'B.test2' on Jul 24 2013 - 14:15:04test2- Finished 'B.test2', execution time = 2.001s

The first decorator class is the reverse method of the class. A similar decorator can be said to be quite useful, implement one of _ lt _, _ le _, _ gt _, _ ge _, and _ eq, can classes be fully sorted? This is the work done by functools. total_ordering. For more information, see references.
Some examples in Flask

Let's take a look at some interesting decorators used in Flask.

Suppose you want some functions to output warning information at specific call times, for example, in debug mode only. And you don't want every function to be added to the control code, so you can use the decorator to implement it. The work of the decorator defined in app. py of Flask is as follows.
 

def setupmethod(f):  """Wraps a method so that it performs a check in debug mode if the  first request was already handled.  """  def wrapper_func(self, *args, **kwargs):    if self.debug and self._got_first_request:      raise AssertionError('A setup function was called after the '        'first request was handled. This usually indicates a bug '        'in the application where a module was not imported '        'and decorators or other functionality was called too late.\n'        'To fix this make sure to import all your view modules, '        'database models and everything related at a central place '        'before the application starts serving requests.')    return f(self, *args, **kwargs)  return update_wrapper(wrapper_func, f)

Let's look at a more interesting example. This example is the route modifier of Flask, which is defined in the Flask class. Note that the decorator Can Be A Method in the class, using self as the first parameter. The complete code is in app. py. Note that the decorator simply registers the decorated function as a URL handle, which is implemented by calling the add_url_rule function.

def route(self, rule, **options): """A decorator that is used to register a view function for a given URL rule. This does the same thing as :meth:`add_url_rule` but is intended for decorator usage::    @app.route('/')   def index():     return 'Hello World'  For more information refer to :ref:`url-route-registrations`.  :param rule: the URL rule as string :param endpoint: the endpoint for the registered URL rule. Flask         itself assumes the name of the view function as         endpoint :param options: the options to be forwarded to the underlying         :class:`~werkzeug.routing.Rule` object. A change         to Werkzeug is handling of method options. methods         is a list of methods this rule should be limited         to (`GET`, `POST` etc.). By default a rule         just listens for `GET` (and implicitly `HEAD`).         Starting with Flask 0.6, `OPTIONS` is implicitly         added and handled by the standard request handling. """ def decorator(f):   endpoint = options.pop('endpoint', None)   self.add_url_rule(rule, endpoint, f, **options)   return f return decorator

Additional reading

1. official Python Wiki

2. metaprogramming in Python 3
Appendix: Closure

A function closure is a combination of a function and a reference set. This reference set points to the variable in the scope defined by this function. The latter usually points to a referencing environment, which allows the function to be executed outside the defined area. In Python, this reference environment is stored in the tuple of a cell. You can access it through the _ closure _ attribute in func_closure or Python 3. One thing to remember is reference and reference, rather than object deep copy. Of course, this is not a problem for immutable objects. However, you must pay attention to the variable Object list. An example will be provided later. Note that the _ globals _ field is also defined in the function to store the global reference environment.

Let's look at a simple example:
 

>>> def return_func_that_prints_s(s):...   def f():...       print s...   return f...>>> g = return_func_that_prints_s("Hello")>>> h = return_func_that_prints_s("World")>>> g()Hello>>> h()World>>> g is hFalse>>> h.__closure__(,)>>> print [str(c.cell_contents) for c in g.__closure__]['Hello']>>> print [str(c.cell_contents) for c in h.__closure__]['World']

A slightly complex example. Make sure that you understand why this is done.
 

>>> def return_func_that_prints_list(z):...   def f():...       print z...   return f...>>> z = [1, 2]>>> g = return_func_that_prints_list(z)>>> g()[1, 2]>>> z.append(3)>>> g()[1, 2, 3]>>> z = [1]>>> g()[1, 2, 3]

Translator: z. when append (3) is used, the reference and z inside g () still point to a variable, and after z = [1], the two no longer point to a variable.

Finally, let's take a look at the definition of the dump_closure method used in the code.
 

def dump_closure(f):  if hasattr(f, "__closure__") and f.__closure__ is not None:    print "- Dumping function closure for %s:" % f.__name__    for i, c in enumerate(f.__closure__):      print "-- cell %d = %s" % (i, c.cell_contents)  else:    print " - %s has no closure!" % f.__name__