Functional programming for Python adorners _python

The Python decorator's name is decorator, and when you see the English name, you might confuse it with the decorator in design, which is a totally different two thing. Although it seems that they want to do is very similar--all want to do some "cosmetic work" for an existing module, the so-called cosmetic work is to add some small decoration (some small functions, these small functions may be used in many modules), But not to allow this small decoration (small function) into the original module in the code. But Oo's decorator is a nightmare, and if you don't believe it, look at the UML diagram and the code in the Wikipedia entry (decorator pattern), which is what I said in the "Dessert" section of "looking at software design from an object-oriented design model". OO encourages-"thick glue and complexity", and "oo enthusiasts are very afraid of processing data" as described in "so understanding object-oriented programming", and decorator's code is simply the reverse course of OO.

Python's decorator is similar to java/c# 's annotation in its use, which is to decorate something for this method by adding a @xxx annotation to the method name. However, Java/c# 's annotation is also very daunting, too TMD complex, you want to play it, you need to know a bunch of annotation library documents, let people feel is learning another language.

Python uses a very elegant method relative to decorator pattern and annotation, which does not require you to master any complex OO models or the various class library rules of annotation. It's all about the language level: a function-programming technique. If you've seen the functional programming of this site, you'll be happy to be programmed to "describe what you want to do, rather than describe how you want to implement it." (If you don't understand functional programming, then ask you to take a look at "functional programming" before reading this article) OK, let's start with some perceptual knowledge, and look at the code for Hello World of the Python modifier.

Hello World

Here's the code:

FileName: hello.py

Copy Code code as follows:

def hello (FN):
def wrapper ():
Print "Hello,%s"% fn.__name__
FN ()
Print "Goodby,%s"% fn.__name__
Return wrapper
@hello
def foo ():
print "I am foo"
Foo ()

When you run the code, you will see the following output:

Copy Code code as follows:

[chenaho@chenhao-air]$ python hello.py
Hello, foo
I am foo
Goodby, foo

You can see the following things:

1 function foo is preceded by a @hello "annotation", Hello is the function we defined earlier Hello

2 in the Hello function, it needs a fn parameter (which is used to do the callback function)

3 The Hello function returns a inner function wrapper, the wrapper function recalls the incoming FN and adds two statements before and after the callback.

The essence of Decorator
For this @ annotation syntax candy-syntactic Sugar in Python, when you are using a @decorator to decorate a function func, as follows:

Copy Code code as follows:

@decorator
def func ():
Pass

Its interpreter is interpreted as the following statement:

Copy Code code as follows:

Func = Decorator (func)

NI, this is not a function when the argument to another function, and then callback? Yes, but we need to be aware that there is also an assignment statement that assigns the return value of the decorator function back to the original func. You can use a function as a variable, according to the definition in the functions in the "function programming", so decorator must return a function to Func, which is called the higher order function High-order Otherwise, there will be an error after the Func () call. As for the example in the hello.py above,

Copy Code code as follows:

@hello
def foo ():
print "I am foo"

was interpreted as:

Copy Code code as follows:

foo = Hello (foo)

Yes, it's a statement, and it's been executed. If you do not believe, you can write this program to try:

Copy Code code as follows:

def fuck (FN):
print "Fuck%s!"% fn.__name__[::-1].upper ()
@fuck
Def WFG ():
Pass

No, the above code, no call WFG () statement, you will find that the fuck function is called, but also very NB of the output of each of our voices!

Back to the example we hello.py, we can see that Hello (foo) returns the wrapper () function, so foo actually becomes a variable of wrapper, and the following foo () execution actually becomes wrapper ().

Knowing this essence, you will not be afraid when you see multiple decorator or decorator with parameters.

For example: multiple decorator

Copy Code code as follows:

@decorator_one
@decorator_two
def func ():
Pass

Equivalent:

Copy Code code as follows:

Func = Decorator_one (Decorator_two (func))

For example: Decorator with parameters:

Copy Code code as follows:

@decorator (Arg1, arg2)
def func ():
Pass

Equivalent:

Copy Code code as follows:

Func = Decorator (arg1,arg2) (func)

This means that the decorator (arg1, arg2) function needs to return a "real decorator".

With parameters and multiple Decrorator
Let's take a look at a few interesting examples:

html.py

Copy Code code as follows:

def makehtmltag (tag, *args, **kwds):


def real_decorator (FN):


Css_class = "class= ' {0} '" ". Format (kwds[" Css_class "]) \


If "Css_class" in Kwds Else ""


def wrapped (*args, **kwds):


Return "<" +tag+css_class+ ">" + fn (*args, **kwds) + "</" +tag+ ">"


return wrapped


Return Real_decorator


@makeHtmlTag (tag= "B", css_class= "Bold_css")


@makeHtmlTag (tag= "i", css_class= "Italic_css")


def hello ():


Return to "Hello World"


Print Hello ()


Output


# <b class= ' bold_css ' ><i class= ' italic_css ' >hello world</i></b>

In the example above, we can see that the Makehtmltag has two parameters. Therefore, in order for Hello = Makehtmltag (arg1, arg2) (hello) to succeed, Makehtmltag must return a decorator (which is why we have added Makehtmltag to Real_decorator ( ), so that we can go into the logic of decorator and--decorator have to return to a wrapper,wrapper to callback hello. It seems that the Makehtmltag () is written in layers, but, has understood the essence of our feel that the writing is very natural.

You see, Python's decorator is so simple, there is nothing complex, you do not need to know too much, use is so natural, thoughtful, dry, breathable, unique quick-impact groove and perfect absorption trajectory, so that you can no longer be anxious for the days of the month and anxiety, Plus the intimate wing design, the quantity is not careful. Sorry, I was naughty.

What, you think there's too much nesting on the top of the decorator with parameters, you can't stand it. OK, OK, let's look at the following method.

Class-style Decorator
First, let's say the class style of decorator, or look at an example:

Copy Code code as follows:

Class Mydecorator (object):
def __init__ (self, FN):
Print "Inside mydecorator.__init__ ()"
Self.fn = fn
def __call__ (self):
Self.fn ()
Print "Inside mydecorator.__call__ ()"
@myDecorator
Def afunction ():
Print "Inside Afunction ()"
Print "Finished decorating afunction ()"
Afunction ()
Output
# inside Mydecorator.__init__ ()
# Finished decorating afunction ()
# inside Afunction ()
# inside Mydecorator.__call__ ()

The example above shows the declaration of a decorator in a class manner. We can see that there are two members in this class:
1 One is __init__ (), this method is called when we give a function decorator, so we need to have a FN parameter, which is the decorator function.
2 One is __call__ (), this method is called when we call the decorator function.
The above output can see the execution order of the entire program.

This looks more readable than the "functional" approach.

Next, let's look at the code that overrides the html.py in a class way:

html.py

Copy Code code as follows:

Class Makehtmltagclass (object):


def __init__ (self, tag, css_class= ""):


Self._tag = Tag


Self._css_class = "class= ' {0} '". Format (Css_class) \


If Css_class!= "" Else ""


def __call__ (self, FN):


def wrapped (*args, **kwargs):


Return "<" + Self._tag + self._css_class+ ">" \


+ FN (*args, **kwargs) + "</" + Self._tag + ">"


return wrapped


@makeHtmlTagClass (tag= "B", css_class= "Bold_css")


@makeHtmlTagClass (tag= "i", css_class= "Italic_css")


def hello (name):


Return "Hello, {}". Format (name)


Print Hello ("Hao Chen")

In the above code, we need to note these points:
1 if decorator has parameters, the __init__ () member cannot pass in FN, and FN is passed in __call__.
2 This code also shows the wrapped (*args, **kwargs) method to pass the parameters of the decorator function. (Where: args is a parameter list, Kwargs is the parameter dict, specific details, please refer to the Python documentation or StackOverflow This problem, this is not the case)

To set the invocation parameters of a function with decorator
There are three ways you can do this:

First, through **kwargs, this method decorator inject parameters into the Kwargs.

Copy Code code as follows:

def decorate_a (function):
def wrap_function (*args, **kwargs):
kwargs[' str '] = ' hello! '
return function (*args, **kwargs)
Return wrap_function
@decorate_A
def print_message_a (*args, **kwargs):
Print (kwargs[' str '])
Print_message_a ()

The second, the appointment of good parameters, directly modify the parameters

Copy Code code as follows:

def decorate_b (function):
def wrap_function (*args, **kwargs):
str = ' hello! '
return function (str, *args, **kwargs)
Return wrap_function
@decorate_B
def print_message_b (str, *args, **kwargs):
Print (str)
Print_message_b ()

Third, injected by *args

Copy Code code as follows:

def decorate_c (function):
def wrap_function (*args, **kwargs):
str = ' hello! '
#args. Insert (1, str)
args = args + (str,)
return function (*args, **kwargs)
Return wrap_function
Class Printer:
@decorate_C
def print_message (self, str, *args, **kwargs):
Print (str)
p = Printer ()
P.print_message ()

Side effects of decorator
here, I believe you should understand the whole Python decorator principle.

Believe you will also find that the decorator function is actually another function, for the first hello.py example, if you query foo.__name__, you will find that the output is "wrapper", rather than our expected "foo", This will bury some holes in our program. So, Python's Functool package provides a decorator called Wrap to eliminate such side effects. Here is our new version of hello.py.

FileName: hello.py

Copy Code code as follows:

From Functools Import Wraps
def hello (FN):
@wraps (FN)
def wrapper ():
Print "Hello,%s"% fn.__name__
FN ()
Print "Goodby,%s"% fn.__name__
Return wrapper
@hello
def foo ():
"' Foo help Doc '"
print "I am foo"
Pass
Foo ()
Print foo.__name__ #输出 foo
Print foo.__doc__ #输出 foo help doc

Of course, even if you use the Functools wraps, you can not completely eliminate such side effects.

Look at the following example:

Copy Code code as follows:

From inspect import getmembers, Getargspec
From Functools Import Wraps
Def wraps_decorator (f):
@wraps (f)
def wraps_wrapper (*args, **kwargs):
Return f (*args, **kwargs)
Return Wraps_wrapper
Class SomeClass (object):
@wraps_decorator
Def method (self, x, y):
Pass
obj = SomeClass ()
For name, func in GetMembers (obj, Predicate=inspect.ismethod):
print ' member name:%s '% name
print ' Func Name:%s '% func.func_name
Print "Args:%s"% Getargspec (func) [0]
Output
# member Name:method
# Func Name:method
# Args: []

You will find that even if you use the Functools wraps, you are missing the parameters when you use Getargspec.

To fix this, we also have to use the reflection of Python to solve, the following is the relevant code:

Copy Code code as follows:

Def get_true_argspec (method):
Argspec = Inspect.getargspec (method)
args = Argspec[0]
If args and args[0] = = ' self ':
Return Argspec
If Hasattr (method, ' __func__ '):
method = method.__func__
If not hasattr (method, ' func_closure ') or method.func_closure is None:
Raise Exception ("No closure for method.")
method = Method.func_closure[0].cell_contents
Return Get_true_argspec (method)

Of course, I'm sure most people's programs don't go to Getargspec. Therefore, the use of functools wraps should be enough.

Some examples of decorator
OK, now let's take a look at the various decorator examples:

Caching function calls
This example is really too classic, the whole online Use this example to do decorator classic example, because is too classic, so, I this article also cannot exception.

Copy Code code as follows:

From Functools Import Wraps
Def memo (FN):
cache = {}
Miss = Object ()
@wraps (FN)
def wrapper (*args):
result = Cache.get (args, Miss)
If result is miss:
result = FN (*args)
Cache[args] = result
return result
Return wrapper
@memo
def fib (n):
If n < 2:
return n
return fib (n-1) + fib (n-2)

In the example above, it is a recursive algorithm for a Fibonacci number example. We know that this recursion is quite inefficient because it will be called repeatedly. For example: We want to compute FIB (5), which is decomposed into fib (4) + FIB (3), while FIB (4) is decomposed into fib (3) +FIB (2), FIB (3) is decomposed into fib (2) +fib (1) ... As you can see, basically, fib (3), FIB (2), FIB (1) are called two times throughout the recursive process.

And we use decorator to query the cache before calling the function, and if not, we can return the value from the cache. All of a sudden, this recursive recursion from a two-fork tree to a linear recursion.

Examples of Profiler
There is nothing profound about this example, it is practical.

Copy Code code as follows:

Import CProfile, pstats, Stringio


Def Profiler (func):


def wrapper (*args, **kwargs):


Datafn = func.__name__ + ". Profile" # Name the data file


Prof = Cprofile.profile ()


retval = Prof.runcall (func, *args, **kwargs)


#prof. Dump_stats (DATAFN)


s = Stringio.stringio ()


SortBy = ' cumulative '


PS = pstats. Stats (Prof, Stream=s). Sort_stats (SortBy)


Ps.print_stats ()


Print S.getvalue ()


return retval


Return wrapper

Registering callback Functions

The following example shows an example of a function that invokes a related registration via the path of a URL:

Copy Code code as follows:

Class MYAPP ():


def __init__ (self):


Self.func_map = {}


def register (self, name):


def func_wrapper (func):


Self.func_map[name] = Func


return func


Return Func_wrapper


def call_method (self, Name=none):


Func = Self.func_map.get (name, None)


If Func is None:


Raise Exception ("No function registered against-" + STR (name))


return func ()


App = MyApp ()


@app. Register ('/')


Def main_page_func ():


Return ' This is the main page. '


@app. Register ('/next_page ')


Def next_page_func ():


Return ' This is the next page. '


Print App.call_method ('/')


Print App.call_method ('/next_page ')

Attention:
1 in the above example, use an instance of the class to do the decorator.
2) There is no __call__ () in the Decorator class, but wrapper returns the original function. Therefore, the original function has not changed anything.

Log to a function

The following example shows a logger decorator, which outputs a function name, parameter, return value, and elapsed time.

Copy Code code as follows:

From Functools Import Wraps


def logger (FN):


@wraps (FN)


def wrapper (*args, **kwargs):


ts = Time.time ()


result = FN (*args, **kwargs)


Te = Time.time ()


print ' function = {0} '. Format (fn.__name__)


print "arguments = {0} {1}". Format (args, Kwargs)


print ' return = {0} '. Format (Result)


Print "Time =%.6f sec"% (te-ts)


return result


Return wrapper


@logger


def multipy (x, y):


return x * y


@logger


def sum_num (n):


s = 0


For I in Xrange (n+1):


s + = i


return s


Print multipy (2, 10)


Print Sum_num (100)


Print Sum_num (10000000)

The above log is still a bit rough, let's look at a better one (with log level parameters):

Copy Code code as follows:

Import Inspect


Def get_line_number ():


Return Inspect.currentframe (). F_back.f_back.f_lineno


def logger (loglevel):


def log_decorator (FN):


@wraps (FN)


def wrapper (*args, **kwargs):


ts = Time.time ()


result = FN (*args, **kwargs)


Te = Time.time ()


print "function =" + fn.__name__,


print "arguments = {0} {1}". Format (args, Kwargs)


print ' return = {0} '. Format (Result)


Print "Time =%.6f sec"% (te-ts)


if (loglevel = = ' Debug '):


Print "Called_from_line:" + str (get_line_number ())


return result


Return wrapper


Return Log_decorator

But there are two bad places on the top with log level parameters,
1 loglevel is not a debug time, or to calculate the time of the function call.
2 different level to write together, not easy to read.

We'll go on to improve:

Copy Code code as follows:

Import Inspect


def advance_logger (loglevel):


Def get_line_number ():


Return Inspect.currentframe (). F_back.f_back.f_lineno


Def _basic_log (FN, result, *args, **kwargs):


print "function =" + fn.__name__,


print "arguments = {0} {1}". Format (args, Kwargs)


print ' return = {0} '. Format (Result)


def info_log_decorator (FN):


@wraps (FN)


def wrapper (*args, **kwargs):


result = FN (*args, **kwargs)


_basic_log (FN, result, args, Kwargs)


Return wrapper


def debug_log_decorator (FN):


@wraps (FN)


def wrapper (*args, **kwargs):


ts = Time.time ()


result = FN (*args, **kwargs)


Te = Time.time ()


_basic_log (FN, result, args, Kwargs)


Print "Time =%.6f sec"% (te-ts)


Print "Called_from_line:" + str (get_line_number ())


Return wrapper


If LogLevel is "debug":


Return Debug_log_decorator


Else


Return Info_log_decorator

You can see two points,
1 We have two log level, one is info, one is debug, and then we return different decorator in the outer end according to different parameters.
2 we drew the same code in info and debug into a function called _basic_log, the dry principle.

A MySQL decorator.
The following decorator is the code I used in my work, and I simplified it by removing the DB connection pool code, which makes it easier to read.

Copy Code code as follows:

Import Umysql


From Functools Import Wraps


Class Configuraion:


def __init__ (self, env):


If env = = "Prod":


Self.host = "coolshell.cn"


Self.port = 3306


self.db = "Coolshell"


Self.user = "Coolshell"


SELF.PASSWD = "FUCKGFW"


elif env = = "Test":


Self.host = ' localhost '


Self.port = 3300


Self.user = ' Coolshell '


self.db = ' Coolshell '


self.passwd = ' FUCKGFW '


def mysql (SQL):


_conf = Configuraion (env= "Prod")


def on_sql_error (err):


Print Err


Sys.exit (-1)


Def handle_sql_result (RS):


If rs.rows &gt; 0:


FieldNames = [f[0] for F in Rs.fields]


return [Dict (Zip (FieldNames, R)) for R in Rs.rows]


Else


return []


def decorator (FN):


@wraps (FN)


def wrapper (*args, **kwargs):


Mysqlconn = Umysql. Connection ()


Mysqlconn.settimeout (5)


Mysqlconn.connect (_conf.host, _conf.port, _conf.user, \


_CONF.PASSWD, _conf.db, True, ' UTF8 ')


Try


rs = mysqlconn.query (sql, {})


Except Umysql. Error as E:


On_sql_error (e)


Data = Handle_sql_result (RS)


kwargs["Data" = Data


result = FN (*args, **kwargs)


Mysqlconn.close ()


return result


Return wrapper


return decorator


@mysql (sql = "SELECT * from Coolshell")


def get_coolshell (data):


... ...


... ..

Thread Asynchronous

The following is a very simple decorator of asynchronous execution, and note that asynchronous processing is not simple, and the following is just an example.

Copy Code code as follows:

From threading Import Thread


From Functools Import Wraps


def async (func):


@wraps (func)


def async_func (*args, **kwargs):


FUNC_HL = Thread (target = func, args = args, Kwargs = Kwargs)


Func_hl.start ()


Return FUNC_HL


Return Async_func


if __name__ = = ' __main__ ':


From time import sleep


@async


Def print_somedata ():


print ' Starting Print_somedata '


Sleep (2)


Print ' Print_somedata:2 sec passed '


Sleep (2)


Print ' Print_somedata:2 sec passed '


Sleep (2)


print ' Finished Print_somedata '


def main ():


Print_somedata ()


print ' Back in main '


Print_somedata ()


print ' Back in main '


Main ()

Although this article is very long, but are very practical, very basic knowledge, hope that the small partners can be patient to finish.