Detailed Python modifier decorator functional Programming "recommendation"

Python's decorator is similar to java/c# 's annotation in its use, which is to decorate something for this method by adding a @xxx annotation to the method name. However, Java/c# 's annotation is also very daunting, too TMD complex, you want to play it, you need to know a bunch of annotation library documents, let people feel is learning another language.





Python uses a very elegant method relative to decorator pattern and annotation, which does not require you to master any complex OO models or the various class library rules of annotation. It's all about the language level: a function-programming technique. If you've seen the functional programming of this site, you'll be happy to be programmed to "describe what you want to do, rather than describe how you want to implement it." (If you don't understand functional programming, then ask you to take a look at "functional programming" before reading this article) OK, let's start with some perceptual knowledge, and look at the code for Hello World of the Python modifier.





Hello World





Here's the code:


FileName: hello.py





def hello (FN):


def wrapper ():


Print "Hello,%s"% fn.__name__


FN ()


Print "Goodby,%s"% fn.__name__


Return wrapper





@hello


def foo ():


print "I am foo"





Foo ()





When you run the code, you will see the following output:





[chenaho@chenhao-air]$ python hello.py


Hello, foo


I am foo


Goodby, foo





You can see the following things:





1 function foo is preceded by a @hello "annotation", Hello is the function we defined earlier Hello





2 in the Hello function, it needs a fn parameter (which is used to do the callback function)





3 The Hello function returns a inner function wrapper, the wrapper function recalls the incoming FN and adds two statements before and after the callback.

the essence of decorator





For this @ annotation syntax candy-syntactic Sugar in Python, when you are using a @decorator to decorate a function func, as follows:





@decorator


def func ():


Pass





Its interpreter is interpreted as the following statement:





Func = Decorator (func)





NI, this is not a function when the argument to another function, and then callback? Yes, but we need to be aware that there is also an assignment statement that assigns the return value of the decorator function back to the original func. You can use a function as a variable, according to the definition in the functions in the "function programming", so decorator must return a function to Func, which is called the higher order function High-order Otherwise, there will be an error after the Func () call. As for the example in the hello.py above,





@hello


def foo ():


print "I am foo"





was interpreted as:





foo = Hello (foo)





Yes, it's a statement, and it's been executed. If you do not believe, you can write this program to try:





def fuck (FN):


print "Fuck%s!"% fn.__name__[::-1].upper ()





@fuck


Def WFG ():


Pass





No, the above code, no call WFG () statement, you will find that the fuck function is called, but also very NB of the output of each of our voices!





Back to the example we hello.py, we can see that Hello (foo) returns the wrapper () function, so foo actually becomes a variable of wrapper, and the following foo () execution actually becomes wrapper ().





Knowing this essence, you will not be afraid when you see multiple decorator or decorator with parameters.





For example: multiple decorator





@decorator_one


@decorator_two


def func ():


Pass





Equivalent:





Func = Decorator_one (Decorator_two (func))





For example: Decorator with parameters:





@decorator (Arg1, arg2)


def func ():


Pass





Equivalent:





Func = Decorator (arg1,arg2) (func)





This means that the decorator (arg1, arg2) function needs to return a "real decorator".

with parameters and multiple Decrorator





Let's take a look at a few interesting examples:


html.py





def makehtmltag (tag, *args, **kwds):


def real_decorator (FN):


Css_class = "class= ' {0} '" ". Format (kwds[" Css_class "])


If "Css_class" in Kwds Else ""


def wrapped (*args, **kwds):


Return "<" +tag+css_class+ ">" + fn (*args, **kwds) + "</" +tag+ ">"


return wrapped


Return Real_decorator





@makeHtmlTag (tag= "B", css_class= "Bold_css")


@makeHtmlTag (tag= "i", css_class= "Italic_css")


def hello ():


Return to "Hello World"





Print Hello ()





Output


# <b class= ' bold_css ' ><i class= ' italic_css ' >hello world</i></b>





In the example above, we can see that the Makehtmltag has two parameters. Therefore, in order for Hello = Makehtmltag (arg1, arg2) (hello) to succeed, Makehtmltag must return a decorator (which is why we have added Makehtmltag to Real_decorator ( ), so that we can go into the logic of decorator and--decorator have to return to a wrapper,wrapper to callback hello. It seems that the Makehtmltag () is written in layers, but, has understood the essence of our feel that the writing is very natural.





You see, Python's decorator is so simple, there is nothing complex, you do not need to know too much, use is so natural, thoughtful, dry, breathable, unique quick-impact groove and perfect absorption trajectory, so that you can no longer be anxious for the days of the month and anxiety, Plus the intimate wing design, the quantity is not careful. Sorry, I was naughty.





What, you think there's too much nesting on the top of the decorator with parameters, you can't stand it. OK, OK, let's look at the following method.

class-style decorator





First, let's say the class style of decorator, or look at an example:





Class Mydecorator (object):





def __init__ (self, FN):


Print "Inside mydecorator.__init__ ()"


Self.fn = fn





def __call__ (self):


Self.fn ()


Print "Inside mydecorator.__call__ ()"





@myDecorator


Def afunction ():


Print "Inside Afunction ()"





Print "Finished decorating afunction ()"





Afunction ()





Output


# inside Mydecorator.__init__ ()


# Finished decorating afunction ()


# inside Afunction ()


# inside Mydecorator.__call__ ()





The example above shows the declaration of a decorator in a class manner. We can see that there are two members in this class:


1 One is __init__ (), this method is called when we give a function decorator, so we need to have a FN parameter, which is the decorator function.


2 One is __call__ (), this method is called when we call the decorator function.


The above output can see the execution order of the entire program.





This looks more readable than the "functional" approach.





Next, let's look at the code that overrides the html.py in a class way:


html.py





Class Makehtmltagclass (object):





def __init__ (self, tag, css_class= ""):


Self._tag = Tag


Self._css_class = "class= ' {0} '". Format (Css_class)


If Css_class!= "" Else ""





def __call__ (self, FN):


def wrapped (*args, **kwargs):


Return "<" + Self._tag + self._css_class+ ">"


+ FN (*args, **kwargs) + "</" + Self._tag + ">"


return wrapped





@makeHtmlTagClass (tag= "B", css_class= "Bold_css")


@makeHtmlTagClass (tag= "i", css_class= "Italic_css")


def hello (name):


Return "Hello, {}". Format (name)





Print Hello ("Hao Chen")





In the above code, we need to note these points:


1 if decorator has parameters, the __init__ () member cannot pass in FN, and FN is passed in __call__.


2 This code also shows the wrapped (*args, **kwargs) method to pass the parameters of the decorator function. (Where: args is a parameter list, Kwargs is the parameter dict, specific details, please refer to the Python documentation or StackOverflow This problem, this is not the case)

to set the invocation parameters of a function with decorator





There are three ways you can do this:





First, through **kwargs, this method decorator inject parameters into the Kwargs.





def decorate_a (function):


def wrap_function (*args, **kwargs):


kwargs[' str '] = ' hello! '


return function (*args, **kwargs)


Return wrap_function





@decorate_A


def print_message_a (*args, **kwargs):


Print (kwargs[' str '])





Print_message_a ()





The second, the appointment of good parameters, directly modify the parameters





def decorate_b (function):


def wrap_function (*args, **kwargs):


str = ' hello! '


return function (str, *args, **kwargs)


Return wrap_function





@decorate_B


def print_message_b (str, *args, **kwargs):


Print (str)





Print_message_b ()





Third, injected by *args





def decorate_c (function):


def wrap_function (*args, **kwargs):


str = ' hello! '


#args. Insert (1, str)


args = args + (str,)


return function (*args, **kwargs)


Return wrap_function





Class Printer:


@decorate_C


def print_message (self, str, *args, **kwargs):


Print (str)





p = Printer ()


P.print_message ()

side effects of decorator





Here, I believe you should understand the whole Python decorator principle.





Believe you will also find that the decorator function is actually another function, for the first hello.py example, if you query foo.__name__, you will find that the output is "wrapper", rather than our expected "foo", This will bury some holes in our program. So, Python's Functool package provides a decorator called Wrap to eliminate such side effects. Here is our new version of hello.py.


FileName: hello.py





From Functools Import Wraps


def hello (FN):


@wraps (FN)


def wrapper ():


Print "Hello,%s"% fn.__name__


FN ()


Print "Goodby,%s"% fn.__name__


Return wrapper





@hello


def foo ():


"' Foo help Doc '"


print "I am foo"


Pass





Foo ()


Print foo.__name__ #输出 foo


Print foo.__doc__ #输出 foo help doc





Of course, even if you use the Functools wraps, you can not completely eliminate such side effects.





Look at the following example:





From inspect import getmembers, Getargspec


From Functools Import Wraps





Def wraps_decorator (f):


@wraps (f)


def wraps_wrapper (*args, **kwargs):


Return f (*args, **kwargs)


Return Wraps_wrapper





Class SomeClass (object):


@wraps_decorator


Def method (self, x, y):


Pass





obj = SomeClass ()


For name, func in GetMembers (obj, Predicate=inspect.ismethod):


print ' member name:%s '% name


print ' Func Name:%s '% func.func_name


Print "Args:%s"% Getargspec (func) [0]





Output


# member Name:method


# Func Name:method


# Args: []





You will find that even if you use the Functools wraps, you are missing the parameters when you use Getargspec.





To fix this, we also have to use the reflection of Python to solve, the following is the relevant code:





Def get_true_argspec (method):


Argspec = Inspect.getargspec (method)


args = Argspec[0]


If args and args[0] = = ' self ':


Return Argspec


If Hasattr (method, ' __func__ '):


method = method.__func__


If not hasattr (method, ' func_closure ') or method.func_closure is None:


Raise Exception ("No closure for method.")





method = Method.func_closure[0].cell_contents


Return Get_true_argspec (method)





Of course, I'm sure most people's programs don't go to Getargspec. Therefore, the use of functools wraps should be enough.

Some examples of decorator

OK, now let's take a look at the various decorator examples:

caching function calls





This example is really too classic, the whole online Use this example to do decorator classic example, because is too classic, so, I this article also cannot exception.





From Functools Import Wraps


Def memo (FN):


cache = {}


Miss = Object ()





@wraps (FN)


def wrapper (*args):


result = Cache.get (args, Miss)


If result is miss:


result = FN (*args)


Cache[args] = result


return result





Return wrapper





@memo


def fib (n):


If n < 2:


return n


return fib (n-1) + fib (n-2)





In the example above, it is a recursive algorithm for a Fibonacci number example. We know that this recursion is quite inefficient because it will be called repeatedly. For example: We want to compute FIB (5), which is decomposed into fib (4) + FIB (3), while FIB (4) is decomposed into fib (3) +FIB (2), FIB (3) is decomposed into fib (2) +fib (1) ... As you can see, basically, fib (3), FIB (2), FIB (1) are called two times throughout the recursive process.





And we use decorator to query the cache before calling the function, and if not, we can return the value from the cache. All of a sudden, this recursive recursion from a two-fork tree to a linear recursion.

Examples of profiler





There is nothing profound about this example, it is practical.





Import CProfile, pstats, Stringio





Def Profiler (func):


def wrapper (*args, **kwargs):


Datafn = func.__name__ + ". Profile" # Name the data file


Prof = Cprofile.profile ()


retval = Prof.runcall (func, *args, **kwargs)


#prof. Dump_stats (DATAFN)


s = Stringio.stringio ()


SortBy = ' cumulative '


PS = pstats. Stats (Prof, Stream=s). Sort_stats (SortBy)


Ps.print_stats ()


Print S.getvalue ()


return retval





Return wrapper

Registering callback Functions





The following example shows an example of a function that invokes a related registration via the path of a URL:





Class MYAPP ():


def __init__ (self):


Self.func_map = {}





def register (self, name):


def func_wrapper (func):


Self.func_map[name] = Func


return func


Return Func_wrapper





def call_method (self, Name=none):


Func = Self.func_map.get (name, None)


If Func is None:


Raise Exception ("No function registered against-" + STR (name))


return func ()





App = MyApp ()





@app. Register ('/')


Def main_page_func ():


Return ' This is the main page. '





@app. Register ('/next_page ')


Def next_page_func ():


Return ' This is the next page. '





Print App.call_method ('/')


Print App.call_method ('/next_page ')





Attention:


1 in the above example, use an instance of the class to do the decorator.


2) There is no __call__ () in the Decorator class, but wrapper returns the original function. Therefore, the original function has not changed anything.

log to a function





The following example shows a logger decorator, which outputs a function name, parameter, return value, and elapsed time.





From Functools Import Wraps


def logger (FN):


@wraps (FN)


def wrapper (*args, **kwargs):


ts = Time.time ()


result = FN (*args, **kwargs)


Te = Time.time ()


print ' function = {0} '. Format (fn.__name__)


print "arguments = {0} {1}". Format (args, Kwargs)


print ' return = {0} '. Format (Result)


Print "Time =%.6f sec"% (te-ts)


return result


Return wrapper





@logger


def multipy (x, y):


return x * y





@logger


def sum_num (n):


s = 0


For I in Xrange (n+1):


s + = i


return s





Print multipy (2, 10)


Print Sum_num (100)


Print Sum_num (10000000)





The above log is still a bit rough, let's look at a better one (with log level parameters):





Import Inspect


Def get_line_number ():


Return Inspect.currentframe (). F_back.f_back.f_lineno





def logger (loglevel):


def log_decorator (FN):


@wraps (FN)


def wrapper (*args, **kwargs):


ts = Time.time ()


result = FN (*args, **kwargs)


Te = Time.time ()


print "function =" + fn.__name__,


print "arguments = {0} {1}". Format (args, Kwargs)


print ' return = {0} '. Format (Result)


Print "Time =%.6f sec"% (te-ts)


if (loglevel = = ' Debug '):


Print "Called_from_line:" + str (get_line_number ())


return result


Return wrapper


Return Log_decorator





But there are two bad places on the top with log level parameters,


1 loglevel is not a debug time, or to calculate the time of the function call.


2 different level to write together, not easy to read.





We'll go on to improve:





Import Inspect





def advance_logger (loglevel):





Def get_line_number ():


Return Inspect.currentframe (). F_back.f_back.f_lineno





Def _basic_log (FN, result, *args, **kwargs):


print "function =" + fn.__name__,


print "arguments = {0} {1}". Format (args, Kwargs)


print ' return = {0} '. Format (Result)





def info_log_decorator (FN):


@wraps (FN)


def wrapper (*args, **kwargs):


result = FN (*args, **kwargs)


_basic_log (FN, result, args, Kwargs)


Return wrapper





def debug_log_decorator (FN):


@wraps (FN)


def wrapper (*args, **kwargs):


ts = Time.time ()


result = FN (*args, **kwargs)


Te = Time.time ()


_basic_log (FN, result, args, Kwargs)


Print "Time =%.6f sec"% (te-ts)


Print "Called_from_line:" + str (get_line_number ())


Return wrapper





If LogLevel is "debug":


Return Debug_log_decorator


Else


Return Info_log_decorator





You can see two points,


1 We have two log level, one is info, one is debug, and then we return different decorator in the outer end according to different parameters.


2 we drew the same code in info and debug into a function called _basic_log, the dry principle.

a MySQL decorator.





The following decorator is the code I used in my work, and I simplified it by removing the DB connection pool code, which makes it easier to read.





Import Umysql


From Functools Import Wraps





Class Configuraion:


def __init__ (self, env):


If env = = "Prod":


Self.host = "coolshell.cn"


Self.port = 3306


self.db = "Coolshell"


Self.user = "Coolshell"


SELF.PASSWD = "FUCKGFW"


elif env = = "Test":


Self.host = ' localhost '


Self.port = 3300


Self.user = ' Coolshell '


self.db = ' Coolshell '


self.passwd = ' FUCKGFW '





def mysql (SQL):





_conf = Configuraion (env= "Prod")





def on_sql_error (err):


Print Err


Sys.exit (-1)





Def handle_sql_result (RS):


If rs.rows > 0:


FieldNames = [f[0] for F in Rs.fields]


return [Dict (Zip (FieldNames, R)) for R in Rs.rows]


Else


return []





def decorator (FN):


@wraps (FN)


def wrapper (*args, **kwargs):


Mysqlconn = Umysql. Connection ()


Mysqlconn.settimeout (5)


Mysqlconn.connect (_conf.host, _conf.port, _conf.user,


_CONF.PASSWD, _conf.db, True, ' UTF8 ')


Try


rs = mysqlconn.query (sql, {})


Except Umysql. Error as E:


On_sql_error (e)





Data = Handle_sql_result (RS)


kwargs["Data" = Data


result = FN (*args, **kwargs)


Mysqlconn.close ()


return result


Return wrapper





return decorator





@mysql (sql = "SELECT * from Coolshell")


def get_coolshell (data):


... ...


... ..

Thread Asynchronous





The following is a very simple decorator of asynchronous execution, and note that asynchronous processing is not simple, and the following is just an example.





From threading Import Thread


From Functools Import Wraps





def async (func):


@wraps (func)


def async_func (*args, **kwargs):


FUNC_HL = Thread (target = func, args = args, Kwargs = Kwargs)


Func_hl.start ()


Return FUNC_HL





Return Async_func





if __name__ = = ' __main__ ':


From time import sleep





@async


Def print_somedata ():


print ' Starting Print_somedata '


Sleep (2)


Print ' Print_somedata:2 sec passed '


Sleep (2)


Print ' Print_somedata:2 sec passed '


Sleep (2)


print ' Finished Print_somedata '





def main ():


Print_somedata ()


print ' Back in main '


Print_somedata ()


print ' Back in main '





Main ()





other





For more examples, you can see: Python decorator Library





For a variety of proposals on Python decroator, see: Python Decorator proposals