Introduce some advanced programming skills in Python

This article mainly introduces some advanced programming skills in Python, including important advanced knowledge points such as the streamer and decorator, which are essential basic skills for deep learning Python development, for more information, see Body:

This article shows some advanced Python design structures and their usage methods. In daily work, you can select an appropriate data structure as needed, such as requirements for fast search, data consistency, and indexing, in addition, various data structures can be properly combined to generate a logical and easy-to-understand data model. The data structure of Python is very intuitive in syntax and provides a large number of optional operations. This guide tries to put most of the commonly used data structure knowledge together and provides a discussion of its best usage.
Comprehensions)

If you have been using Python for a long time, you should have heard of list comprehensions at least ). This is a way to put the for loop, if expression, and value assignment statement into a single statement. In other words, you can map or filter a list using an expression.

A list derivation contains the following parts:

• An input sequence
• A variable that represents the input sequence members.
• An optional asserted expression
• A member that satisfies the assertion expression in the input sequence is transformed into an output expression of the output list member.

For example, we need to generate a new sequence from an input list of all integers greater than 0. you may write this:

num = [1, 4, -5, 10, -7, 2, 3, -1]filtered_and_squared = [] for number in num: if number > 0: filtered_and_squared.append(number ** 2)print filtered_and_squared # [1, 16, 100, 4, 9]

It's easy, right? However, there will be four lines of code, two layers of nesting, plus a completely unnecessary append operation. If the filter, lambda, and map functions are used, the code can be greatly simplified:

num = [1, 4, -5, 10, -7, 2, 3, -1]filtered_and_squared = map(lambda x: x ** 2, filter(lambda x: x > 0, num))print filtered_and_squared # [1, 16, 100, 4, 9]

Well, the code will be expanded horizontally. Can I continue to simplify the code? List derivation gives us the answer:

num = [1, 4, -5, 10, -7, 2, 3, -1]filtered_and_squared = [ x**2 for x in num if x > 0]print filtered_and_squared # [1, 16, 100, 4, 9]

• Iterator traverses each member of the input sequence num x
• Judge whether each member is greater than zero by assertion
• If the member is greater than zero, it is handed over to the output expression, and the square is then the member of the output list.

The list derivation is encapsulated in a list, so it is obvious that it can generate a new list immediately. Here, there is only one type function call without calling lambda functions implicitly. the list derivation uses a regular iterator, an expression, and an if expression to control optional parameters.

On the other hand, list derivation may also have some negative effects, that is, the entire list must be loaded into the memory at a time, which is not a problem for the above example, it is not a problem even after several times of expansion. But it will always reach the limit, and the memory will always be used up.

The Generator can solve the above problems well. The Generator expression does not load the entire list to the memory at a time, but generates a Generator object (Generator objector). Therefore, only one list element is loaded at a time.

The generator expression has almost the same syntax structure as the list derivation. The difference is that the generator expression is surrounded by parentheses rather than square brackets:

num = [1, 4, -5, 10, -7, 2, 3, -1]filtered_and_squared = ( x**2 for x in num if x > 0 )print filtered_and_squared # 
 
   at 0x00583E18> for item in filtered_and_squared: print item # 1, 16, 100 4,9
 

This is a little more efficient than list derivation. Let's transform the code again:

num = [1, 4, -5, 10, -7, 2, 3, -1] def square_generator(optional_parameter): return (x ** 2 for x in num if x > optional_parameter) print square_generator(0)# 
 
   at 0x004E6418> # Option Ifor k in square_generator(0): print k# 1, 16, 100, 4, 9 # Option IIg = list(square_generator(0))print g# [1, 16, 100, 4, 9]
 

Unless otherwise specified, generator expressions are often used in the code. However, unless there is a very large list, there is no obvious difference.

The following example uses the zip () function to process elements in two or more lists at a time:

alist = ['a1', 'a2', 'a3']blist = ['1', '2', '3'] for a, b in zip(alist, blist): print a, b # a1 1# a2 2# a3 3

Let's take a look at an example of directory traversal through list derivation of two levels:

import osdef tree(top): for path, names, fnames in os.walk(top): for fname in fnames:  yield os.path.join(path, fname) for name in tree('C:\Users\XXX\Downloads\Test'): print name

Decorators)

The decorator provides an effective method for adding existing functions or classes. Does it sound like Aspect-Oriented Programming in Java? Both of them are simple, and the decorator has more powerful functions. For example, if you want to perform some special operations (such as security, tracking, and locking) at the entry and exit points of a function, you can use the decorator.

The decorator is a special function that encapsulates another function: the main function is called, and its return value will be passed to the decorator, next, the decorator will return an alternative function that encapsulates the main function. The other part of the program will see this packaging function.

def timethis(func): ''' Decorator that reports the execution time. ''' pass @timethisdef countdown(n): while n > 0: n -= 1

Syntactic sugar @ identifies the decorator.

Let's go back to the example. We will use the decorator for some more typical operations:

import timefrom functools import wraps def timethis(func): ''' Decorator that reports the execution time. ''' @wraps(func) def wrapper(*args, **kwargs): start = time.time() result = func(*args, **kwargs) end = time.time() print(func.__name__, end-start) return result return wrapper @timethisdef countdown(n): while n > 0: n -= 1 countdown(100000) # ('countdown', 0.006999969482421875)

When you write the following code:

@timethisdef countdown(n):

This means that you have performed the following steps separately:

def countdown(n):...countdown = timethis(countdown)

The code in the decorator function creates a new function (just like the wrapper function in this example). it uses * args and ** kwargs to receive arbitrary input parameters, in addition, the original function is called in this function and the result is returned. You can place any additional code as needed (such as timing operations in this example). the newly created packaging function will return the result and replace the original function.

@decoratordef function(): print("inside function")

When the compiler looks at the above code, the function () function will be compiled and the returned object of the function will be passed to the decorator code, the decorator replaces the original function with a new function object after completing related operations.

What is the decorator code? In most cases, the decorator is defined as a function, and I found that it is easier to define the decorator as a class to understand its functions and give full play to the power of the decorator mechanism.

The only requirement for the class implementation of the decorator is that it must be usable as a function, that is, it must be callable. Therefore, to do this, the class must implement the _ call _ method.

What can I do with such a decorator? It can do anything, but it is usually used when you want to use the original function in some special places, but this is not necessary, for example:

class decorator(object):  def __init__(self, f): print("inside decorator.__init__()") f() # Prove that function definition has completed  def __call__(self): print("inside decorator.__call__()") @decoratordef function(): print("inside function()") print("Finished decorating function()") function() # inside decorator.__init__()# inside function()# Finished decorating function()# inside decorator.__call__()

Note:
1. syntax sugar @ decorator is equivalent to function = decorator (function). call decorator's _ init _ to print "inside decorator. _ init __()"
2. then execute f () to print "inside function ()"
3. then run "print (" Finished decorating function ()")"
4. finally, when calling the function, because the decorator is encapsulated, execute decorator's _ call _ to print "inside decorator. _ call __()".

A more practical example:

def decorator(func): def modify(*args, **kwargs): variable = kwargs.pop('variable', None) print variable x,y=func(*args, **kwargs) return x,y return modify @decoratordef func(a,b): print a**2,b**2 return a**2,b**2 func(a=4, b=5, variable="hi")func(a=4, b=5) # hi# 16 25# None# 16 25

ContextLib)

The contextlib module contains tools related to the context manager and with declaration. Generally, if you want to write a context manager, you need to define a class containing the _ enter _ method and the _ exit _ method. for example:

import timeclass demo: def __init__(self, label): self.label = label  def __enter__(self): self.start = time.time()  def __exit__(self, exc_ty, exc_val, exc_tb): end = time.time() print('{}: {}'.format(self.label, end - self.start))

The complete example is as follows:

import time class demo: def __init__(self, label): self.label = label  def __enter__(self): self.start = time.time()  def __exit__(self, exc_ty, exc_val, exc_tb): end = time.time() print('{}: {}'.format(self.label, end - self.start)) with demo('counting'): n = 10000000 while n > 0: n -= 1 # counting: 1.36000013351

The context manager is activated by the with Declaration. This API involves two methods.
1. _ enter _ method. when the execution flow enters the with code block, the __enter _ method will be executed. In addition, it returns an object available for the context.
2. when the execution stream leaves the with code block, the __exit _ method is called and the used resources are cleared.

Use @ contextmanager to rewrite the example above:

from contextlib import contextmanagerimport time @contextmanagerdef demo(label): start = time.time() try: yield finally: end = time.time() print('{}: {}'.format(label, end - start)) with demo('counting'): n = 10000000 while n > 0: n -= 1 # counting: 1.32399988174

In the above example, all the code before yield in the function is similar to the content of The _ enter _ method in the context manager. All the code after yield is like the content of The _ exit _ method. If an exception occurs during execution, the yield statement is triggered.
Descriptors)

The descriptor determines how object properties are accessed. The descriptor is used to customize the operations that occur when you want to reference an attribute.

The method to build the descriptor is to define at least one of the following three methods. Note that the instance in the following section contains the object instance of the Accessed attribute, while the owner is the rhetorical class of the describer.

_ Get _ (self, instance, owner)-This method is used when the attribute is passed (value = obj. attr), the return value of this method is assigned to the code section requesting this attribute value.
_ Set _ (self, instance, value)-This method is used when you want to set the attribute value (obj. attr = 'value') is called. this method does not return any value.
_ Delete _ (self, instance)-This method is called when an attribute (del obj. attr) is deleted from an object.

Note: Consider the following code for understanding the instance and owner:

class Celsius(object): def __init__(self, value=0.0): self.value = float(value) def __get__(self, instance, owner): return self.value def __set__(self, instance, value): self.value = float(value) class Temperature(object): celsius = Celsius() temp=Temperature()temp.celsius #calls Celsius.__get__

In the above example, instance refers to temp, while owner refers to Temperature.

Example of LazyLoading Properties:

import weakref class lazyattribute(object): def __init__(self, f): self.data = weakref.WeakKeyDictionary() self.f = f def __get__(self, obj, cls): if obj not in self.data:  self.data[obj] = self.f(obj) return self.data[obj] class Foo(object): @lazyattribute def bar(self): print "Being lazy" return 42 f = Foo() print f.bar# Being lazy# 42 print f.bar# 42

The descriptor summarizes the concept of bound method in Python. the binding method is the core of classic classes. In a classic class, if an attribute is not found in the Dictionary of an object instance, the Dictionary of the class is searched and then to the Dictionary of the base class, this keeps searching recursively. If this attribute is found in the class dictionary, the interpreter checks whether the object is a Python function object. If yes, the returned object is not the object itself, but a currying function (currying function) package object. When the package is called, it inserts an instance before the parameter list, and then calls the original function.

Note:
1. colihua-http://zh.wikipedia.org/wiki/%E6%9F%AF%E9%87%8C%E5%8C%96
2. differences between function, method, bound method and unbound method. First, a function is created by def or lambda. When a function is defined in the class statement block or created by type, it is converted into an unbound method) when you access this method, it will be converted to the binding method, the binding method will automatically pass the instance as the first parameter to the method. To sum up, the method is a function that appears in the class. the binding method is a method bound to a specific instance, and vice versa.

In summary, the descriptor is assigned to the class, and these special methods are automatically called when the attribute is accessed according to the specific access type.
MetaClasses)

The metadata class provides an effective way to change the behavior of the Python class.

The metadata is defined as a class ". Any instance of its own class is a metadata class.

class demo(object): pass obj = demo() print "Class of obj is {0}".format(obj.__class__)print "Class of obj is {0}".format(demo.__class__) # Class of obj is 
 
  # Class of obj is 
  
 

In the above example, we define a class demo and generate an Object obj for this class. First, we can see that obj _ class _ is a demo. What is interesting? what is the demo class? We can see that the demo's _ class _ is type.

So type is a python class. In other words, obj in the above example is a demo object, and demo itself is a type object.

Therefore, type is a metadata class, which is the most common metadata class in python, because it makes the default metadata of all classes in python.

Because a Metaclass is a class, it is used to create a class (just as a class is used to create an object ). However, aren't we using a standard class definition to create a class? This is true, but the internal operating mechanism of python is as follows:

• When you see a class definition, python collects all the attributes into a dictionary.
• When the class definition ends, python will determine the class Meta class. let's call it Meta.
• Finally, python executes Meta (name, bases, dct), where:

A. Meta is a Meta class, so this call is to instantiate it.
B. name is the class name of the new class.
C. bases is the base class tuples of the new class.
D. dct maps attribute names to objects to list all class attributes.

So how to determine the meta class of A class (? To put it simply, if A class (A) itself or one of its base classes (Base_A) has the _ metaclass _ attribute, the class (A/Base_A) is A class (). Otherwise, type is the metadata of class (.
Patterns)

"Request forgiveness is easier than request authorization (EFAP )"

The Python design principle is that "request forgiveness is easier (EFAP) than request permission )". We do not advocate thoughtful design ideas. this principle is that we should try our best and handle errors properly. Python has a powerful exception handling mechanism to support such attempts. these mechanisms help programmers develop programs with higher stability and fault tolerance.

Singleton

Singleton means that only one instance object can exist at the same time. Python provides many methods to implement Singleton.

Null Object

Null objects can be used to replace the None type to avoid testing None.

Observer

The Observer Mode allows multiple objects to access the same data.

Constructor

Constructor parameters are often assigned to instance variables. This mode can replace multiple manual assignment statements with one line of code.
Summary

Thank you for reading. if you have any questions, please leave a message for discussion.