Detailed tutorial on the adorners, closures and functools in Python _python

Adorners (Decorators)

Adorners are a design pattern that can be used if a class wants to add some functionality to other classes, rather than inheriting or directly modifying the source code implementation. In short, an adorner in Python means some function or other callable object that takes a function or class as an optional input parameter, and then returns the form of a function or class. This feature, which is added in the Python2.6 version, can be used to implement the adorner design pattern.

By the way, if you're not sure about the Closure concept in Python until you continue reading, check out the appendix at the end of this article, and it's hard to understand the adorner in Python properly without the concept of closure.

In Python, an adorner is used in a function or class of the @ syntactic sugar rhetoric. Now let's use a simple decorator example to demonstrate how to do a function call logger. In this example, the adorner takes the time format as an input parameter and prints out the time of the function call when the function decorated by the adorner is invoked. This adorner is useful when you need to manually compare two different algorithms or implementations.

def logged (Time_format):
  def Decorator (func):
   def decorated_func (*args, **kwargs):
     print "-Running '%s ' On%s '% (
                     func.__name__,
                     time.strftime (Time_format)
               )
     start_time = time.time () result
     = Func (* args, **kwargs)
     end_time = Time.time ()
     print "-Finished '%s ', Execution time =%0.3fs"% (
                     func.__name__,< C12/>end_time-start_time
               ) return result
   decorated_func.__name__ = func.__name__
   return Decorated_func return
 Decorator

To see an example where the ADD1 and ADD2 functions are modified by logged, a sample output is given below. Note that here the time format parameter is stored in the returned adorner function (Decorated_func). This is why understanding closures is important for understanding adorners. Also notice how the name of the return function is replaced with the original function name, just in case it is to be used, to prevent confusion. Python does not do this by default.

@logged ("%b%d%Y-%h:%m:%s")
def add1 (x, Y):
  time.sleep (1) return
  x + y
 
@logged ("%b%d%Y-%h:%m:%s") 
   def add2 (x, y):
  time.sleep (2) return
  x + y
 
print add1 (1, 2)
print ADD2 (1, 2)
 
# Output:
-Run Ning ' Add1 ' on June 2013-13:40:47
-finished ' add1 ', execution time = 1.001s
3
-Running ' add2 ' on June 2013-13:40:48
-Finished ' add2 ', execution time = 2.001s
3

If you are careful enough, you may notice that we have a special treatment for the name __name__ of the return function, but not for other injections of __doc__ or __module__. So if, in this example, the Add function has a doc string, it is discarded. So how do you deal with it? We can certainly treat all the fields as we do with __name__, but it's too cumbersome to do so in every decorator. This is why the Functools module provides an adorner named wraps, which is to deal with this situation. may be confusing in understanding the adorner, but when you look at the adorner as an input parameter and return a function, this is well understood. We'll use the wraps adorner in the next example instead of handling __name__ or other properties manually.

The next example is a bit complicated, and our task is to cache the return result of a function call for a period of time, and the input parameter determines the cache time. The input parameter passed to the function must be a hash object, because we use the tuple that contains the calling input parameter as the first argument, the second argument is a Frozenset object, it contains the keyword item Kwargs, and is the cache key. Each function will have a unique cache dictionary stored in the function's closure.

The

"Set" and "Frozenset" are two built-in collections of Python, the former being a mutable object (mutable), whose elements can be changed using Add () or remove (), which are immutable objects (imutable) and are hashed ( hashable), the element is immutable after establishment, and he can be the key of the dictionary or the element of another collection.

Import time from Functools import wraps def cached (timeout, logged=false): "" "decorator to cache the result of a Func
  tion call.
  Cache expires after timeout seconds. "" "Def Decorator (func): If Logged:print"-Initializing cache for ", func.__name__ cache = {} @wra
      PS (func) def decorated_function (*args, **kwargs): If Logged:print "--called function", func.__name__  Key = (args, Frozenset (Kwargs.items ()) result = None if key in Cache:if Logged:print "--Cache hit for", Func.__name__, Key (cache_hit, expiry) = Cache[key] If time.time ()-Expiry < Ti
      Meout:result = Cache_hit elif logged:print "--cache expired for", func.__name__, key Elif Logged:print "-Cache miss for", func.__name__, Key # No Cache hit, or expired if ' N One:result = Func (*args, **kwargs) Cache[key] = (result, Time.time ()) REturn result return decorated_function return decorator to see its usage. We use adorners to decorate a very basic Fibonacci generator. This cache adorner will use Memo mode for code (memoize pattern). Notice how the closure of the FIB function holds the cache dictionary, a reference to the original FIB function, the value of the logged parameter, and the last value of the timeout parameter.

Dump_closure will be defined at the end of the article.   >>> @cached (True) ... def fib (n):.
"" "Returns the n ' th Fibonacci number." "" ... if n = = 0 or n = 1: ... return 1 ... return fib (n-1) + fib (n-2) ...--initializing cache for fib >>> dump_closure (fib) 1. Dumping function closure for fib:--cell 0 = {}--cell 1 =--Cell 2 = True--Cell 3 = >>> >>> pri NT "Testing-f (4) =%d"% fib (4)--called function fib-Cache miss for Fib ((4,), Frozenset ([]))--called function fi  B-Cache miss for Fib ((3,), Frozenset ([]) – Called function fib-cache miss for Fib ((2,), Frozenset ([]))--called function Fib-Cache miss for Fib ((1,), Frozenset ([]) – Called function fib-cache miss for FIB (0,), Frozenset ([] )--called function fib-Cache hit for fiB ((1,), Frozenset ([]))--called function fib-Cache hit for Fib ((2,), Frozenset ([])) testing-f (4) = 5 Class Decorat

 Ors

In the previous section, we looked at some of the function adorners and some of the tips we used, and we'll look at the class adorners. The class adorner takes a class as an input parameter (a class-type object in Python) and returns a modified class.

The first example is a simple mathematical problem. When given an ordered set P, we define the reverse sequence set P (x,y) <-> PD (X,y) of p, which means that the sequence of elements in two ordered sets is reversed, how is this implemented in Python? Suppose a class defines __lt__ and __le__ or other methods to achieve order. Then we can replace these methods by writing a class adorner.

def make_dual (relation):
  @wraps (Relation, [' __name__ ', ' __doc__ '])
  def dual (x, y): return
    relation (y, X) Return
  dual
 
def dual_ordering (CLS): "" "Class decorator that reverses all orderings" "for
  func in
  [' __lt__ ', ' __gt__ ', ' __ge__ ', ' __le__ ']:
    if Hasattr (CLS, func): SetAttr (CLS, func, Make_dual (
      getattr (CLS, func) ) Return
  CLS

The following is an example of the type of STR used for this adorner, creating a new class named RSTR, using the Reverse dictionary order (opposite lexicographic) for the sequence.

@dual_ordering
class Rstr (str):
  pass
 
x = RSTR ("1")
y = Rstr ("2")
 
print x < y
print x <= y
   print x > Y
print x >= y
 
# Output:
false
false
true

Look at a more complicated example. Suppose we want the previous logged adorner to be used for all the methods of a class. One scenario is to add an adorner to each class method. Another scenario is to write a class adorner that automatically completes these tasks. Before I do this, I'll take the logged adorner in the precedent and make some minor improvements. First, it uses the wraps adorner provided by Functools to complete the fixed __name__ work. Second, a _logged_decorator attribute is introduced (a Boolean variable set to TRUE) to indicate whether the method has been decorated by the adorner, as the class may be inherited and the subclass may continue to use the adorner. Finally, the Name_prefix parameter is added to set the log information for printing.

def logged (Time_format, name_prefix= ""):
  def Decorator (func):
    if Hasattr (func, ' _logged_decorator ') and func. _logged_decorator: Return
      func
 
    @wraps (func)
    def decorated_func (*args, **kwargs):
      start_time = Time.time ()
      print "-Running '%s ' on%s"% (
                      Name_prefix + func.__name__,
                      time.strftime (Time_format)
                 ) Result
      = func (*args, **kwargs)
      end_time = Time.time ()
      print "-Finished '%s ', Execution time =%0.3fs"% (
                      Name_prefix + func.__name__,
                      end_time-start_time
                 ) return result
    decorated_func._ Logged_decorator = True return
    decorated_func return
  decorator

OK, let's start writing class adorners:

def log_method_calls (Time_format):
  def decorator (CLS): For
    o in dir (CLS):
      If O.startswith (' __ '):
        Continue
      A = GetAttr (CLS, O)
      if hasattr (A, ' __call__ '):
        decorated_a = Logged (Time_format, cls.__name__ + ".") (a)
        SetAttr (CLS, O, decorated_a) return
    CLS return
  decorator

Here's how to use the method to notice how the inherited or overridden method is handled.

@log_method_calls ("%b%d%Y-%h:%m:%s")
class A (object):
  def test1 (self):
    print "Test1"
 
@log_method_ Calls ("%b%d%Y-%h:%m:%s")
Class B (A):
  def test1 (self):
    super (b, self). Test1 ()
    print "Child test1"
 
  def test2 (self):
    print "Test2"
 
B = B ()
b.test1 ()
b.test2 ()
 
# Output:
-Running ' B.test1 ' on June 2013-14:15:03
-Running ' A.test1 ' on June 2013-14:15:03
test1
-Finished ' A.test1 ' , Execution time = 0.000s child
test1
-Finished ' b.test1 ', execution time = 1.001s
-Running ' b.test2 ' on June 2013-14:15:04
test2
-Finished ' b.test2 ', Execution time = 2.001s

The example of our first class adorner is the reverse ordering method of the class. A similar adorner, can be said to be quite useful, to achieve the __lt__, __le__, __gt__, __ge__ and __eq__ One, can realize the whole class sort? This is also the work done by the functools.total_ordering adorner. See the reference documentation for details.
Some examples in the Flask

Let's look at some of the interesting adorners used in flask.

Suppose you want some functions to output warning messages at a particular call time, for example, only in debug mode. And you don't want every function to add control code, then you can use the adorner to implement it. The following is the work of the adorner defined in Flask's app.py.

Def SetupMethod (f): "" "" "
  wraps a method So, it performs a check in debug mode if the" a "is
  already h andled.
  "" " def wrapper_func (self, *args, **kwargs):
    if Self.debug and self._got_first_request:
      raise Assertionerror (' A Setup function is called after the ' ' ' The ' ' The ' ' The ' ' The ' '
        . This usually indicates a bug "in the
        application where a module is not imported '" and
        decorators or other Func  Tionality is called too late.\n '
        to the fix this make sure to the import all your view modules, '
        database models and Everything related at a-place '
        before the application starts serving. ')
    Return F (Self, *args, **kwargs) return
  update_wrapper (Wrapper_func, F)

To see a more interesting example, this example is the Flask route adorner, defined in the Flask class. Note that the adorner can be a method in a class, and self as the first argument. The complete code is in the app.py. Please note that the adorner simply registers the decorated function as a URL handle, which is achieved by invoking the Add_url_rule function.

def route (self, rule, **options): "" "A decorator this is used to register a view function for a given URL rule. This does the same thing as:meth: ' Add_url_rule ' but are intended for decorator usage:: @app. Route ('/') def index (
 
 ): Return "Hello World" For more information refer to:ref: ' url-route-registrations '. :p Aram rule:the URL rule As String:p Aram Endpoint:the endpoint to the registered URL rule. Flask itself assumes the name of the view function as endpoint:p Aram options:the options to is Forwar Ded to the Underlying:class: ' ~werkzeug.routing.rule ' object. A change to Werkzeug is handling the method options. Methods is a list of methods this rule should being limited to (' Get ', ' POST ' etc.).
         By default A, just listens for ' get ' (and implicitly ' head ').
 Starting with flask 0.6, ' OPTIONS ' are implicitly added and handled by the standard request handling. "" "Def Decorator (f): Endpoint = options.pop (' endpoint ', None) self.add_url_rule (rule, endpoint, F, **options) return F return Decor
 Ator

Extended Reading

1. Official Python Wiki

2. Metaprogramming in Python 3
Appendix: Closures

A function closure is a combination of a function and a reference set, which points to a variable of the scope defined by the function. The latter usually points to a reference environment (referencing environment), which enables the function to execute outside of the area it is defined. In Python, this reference environment is stored in the tuple of a cell. You can access it through the func_closure or the __closure__ attribute in Python 3. One thing to keep in mind is that references and references are not deep copies of objects. Of course, this is not a problem for mutable objects, but it has to be noted for variable objects (list), followed by an example. Note that the function also has a __globals__ field defined where it stores the global reference environment.

Let's look at a simple example:

>>> def return_func_that_prints_s (s):
...   def f ():
.       . Print s ...   Return f ...
>>> g = return_func_that_prints_s ("Hello")
>>> h = return_func_that_prints_s ("World")
>>> g ()
Hello
>>> H ()
World
>>> G is h
False
>>> H . __closure__
(,)
>>> print [str (c.cell_contents) for C in g.__closure__]
[' Hello ']
> >> Print [str (c.cell_contents) for C into h.__closure__]
[' World ']

A slightly more complicated example. Make sure you understand why this is done.

>>> def return_func_that_prints_list (z):
...   def f ():
.       . Print Z
...   Return f ...
>>> z = [1, 2]
>>> g = Return_func_that_prints_list (z)
>>> g ()
[1, 2]
>>> z.append (3)
>>> g ()
[1, 2, 3]
>>> z = [1]
>>> g ()
[1, 2, 3]

"Translator": When Z.append (3), the reference in G () and z still point to a variable, and z=[1], the two no longer point to a variable.

Finally, take a look at the definition of the Dump_closure method used in the code.

Def dump_closure (f):
  if Hasattr (f, "__closure__") and f.__closure__ are not None:
    print-dumping function closure For%s: '% f.__name__
    for I, C in enumerate (f.__closure__):
      print '--cell%d =%s '% (i, c.cell_contents)
  el SE:
    print "-%s has no closure!"% f.__name__