Summary of some traps and tips in Python, and summary of python traps

Summary of some traps and tips in Python, and summary of python traps

Python is a widely used and powerful language that allows us to go deep into this language and learn some control statement skills, standard library tips, and some common traps.

Python (and its various libraries) is very large. It is used for System Automation, web applications, big data, data analysis, and security software. This document is intended to demonstrate a little-known technique that will lead you to the path of faster development, easier debugging, and fun.

Learning Python is the same as learning all other languages. The really useful resource is not a very large official document in every language, but the ability to share knowledge using common syntaxes, libraries, and the Python community.
Explore Standard Data Types
Humble enumerate

Traversal is very simple in Python. You can use "for foo in bar.
 

drinks = ["coffee", "tea", "milk", "water"]for drink in drinks:  print("thirsty for", drink)#thirsty for coffee#thirsty for tea#thirsty for milk#thirsty for water

However, using the sequence numbers and elements is also a common requirement. We often see that some programmers use len () and range () to iterate the list by subscript, but there is a simpler way.
 

drinks = ["coffee", "tea", "milk", "water"]for index, drink in enumerate(drinks):  print("Item {} is {}".format(index, drink))#Item 0 is coffee#Item 1 is tea#Item 2 is milk#Item 3 is water

The enumerate function can traverse elements and their serial numbers at the same time.
Set Type

Many concepts can be attributed to the operations on the set. For example, confirm that a list has no repeated elements, and view the elements of the two lists. Python provides the set data type to make operations like this faster and more readable.
 

# deduplicate a list *fast*print(set(["ham", "eggs", "bacon", "ham"]))# {'bacon', 'eggs', 'ham'} # compare lists to find differences/similarities# {} without "key":"value" pairs makes a setmenu = {"pancakes", "ham", "eggs", "bacon"}new_menu = {"coffee", "ham", "eggs", "bacon", "bagels"} new_items = new_menu.difference(menu)print("Try our new", ", ".join(new_items))# Try our new bagels, coffee discontinued_items = menu.difference(new_menu)print("Sorry, we no longer have", ", ".join(discontinued_items))# Sorry, we no longer have pancakes old_items = new_menu.intersection(menu)print("Or get the same old", ", ".join(old_items))# Or get the same old eggs, bacon, ham full_menu = new_menu.union(menu)print("At one time or another, we've served:", ", ".join(full_menu))# At one time or another, we've served: coffee, ham, pancakes, bagels, bacon, eggs

The intersection function compares all elements in the list and returns the intersection of the two sets. In our example, the staple food for breakfast is bacon, eggs, and ham.
Collections. namedtuple

If you do not want to add a method to a class but want to use the call method of foo. prop, you need namedtuple. You define the class attributes in advance, and then you can instantiate a lightweight class. This method consumes less memory than the complete object.
 

LightObject = namedtuple('LightObject', ['shortname', 'otherprop'])m = LightObject()m.shortname = 'athing'> Traceback (most recent call last):> AttributeError: can't set attribute

In this way, you cannot set the attributes of namedtuple, just as you cannot modify the values of elements in tuple. You need to set the attribute value when instantiating namedtuple.
 

LightObject = namedtuple('LightObject', ['shortname', 'otherprop'])n = LightObject(shortname='something', otherprop='something else')n.shortname # somethingcollections.defaultdict

When writing a Python application to use a dictionary, some keywords do not exist at the beginning, for example, the following example.
 

login_times = {}for t in logins:  if login_times.get(t.username, None):    login_times[t.username].append(t.datetime)  else:    login_times[t.username] = [t.datetime]

With defaultdict, we can skip the logic of checking whether the keyword exists. Any access to an undefined key will return an empty list (or another data type ).
 

login_times = collections.defaultdict(list)for t in logins:  login_times[t.username].append(t.datetime)

You can even use a custom class to instantiate a class during the call.
 

from datetime import datetimeclass Event(object):  def __init__(self, t=None):  if t is None:    self.time = datetime.now()  else:    self.time = t events = collections.defaultdict(Event) for e in user_events:  print(events[e.name].time)

If you want to use the defaultdict feature and access attributes to process nested keys, you can take a look at addict.
 

normal_dict = {  'a': {    'b': {      'c': {        'd': {          'e': 'really really nested dict'        }      }    }  }} from addict import Dictaddicted = Dict()addicted.a.b.c.d.e = 'really really nested'print(addicted)# {'a': {'b': {'c': {'d': {'e': 'really really nested'}}}}}

This small program is much easier to write than the standard dict. So why not use defaultdict? It looks simple enough.
 

from collections import defaultdictdefault = defaultdict(dict)default['a']['b']['c']['d']['e'] = 'really really nested dict' # fails

This code looks okay, But it finally throws a KeyError exception. This is because the default ['a'] Is dict, not defaultdict. Let's construct a defaultdict whose value is defaulted dictionaries. In this way, only two levels of nesting can be solved.

If you only need a default Counter, you can use collection. Counter. This class provides many convenient functions, such as most_common.
Control Flow

When learning the control structure in Python, we usually need to carefully learn for, while, if-elif-else, and try-else t. These control structures can handle the vast majority of cases as long as they are correctly used. For this reason, almost all the languages you encounter provide similar control structure statements. In addition to the basic control structure, Python also provides some uncommon control structures that make your code more readable and maintainable.
Great invocations tations

As a control structure, Exceptions are frequently used to process databases, sockets, files, or any resources that may fail. This is often the case when writing data to a database using a standard try or mongot structure.
 

try:  # get API data  data = db.find(id='foo') # may raise exception  # manipulate the data  db.add(data)  # save it again  db.commit() # may raise exceptionexcept Exception:  # log the failure  db.rollback() db.close()

Can you find the problem here? Two possible exceptions will trigger the same ipvt module. This means that data query failure (or connection failure for data query) will cause a rollback operation. This is definitely not what we want, because the transaction has not started at this point in time. Similarly, rollback should not be the correct response to database connection failure, so let's separate different situations.

First, we will process the query data.

try:  # get API data  data = db.find(id='foo') # may raise exceptionexcept Exception:  # log the failure and bail out  log.warn("Could not retrieve FOO")  return # manipulate the datadb.add(data)

Data retrieval now has its own try-then t, so that when we do not get data, we can take any processing method. Without data, our code is unlikely to do anything useful, so we will just exit the function. In addition to exiting, you can also construct a default object to re-search or end the entire program.

Now let's package the commit Code separately so that it can handle errors more elegantly.
 

try:  db.commit() # may raise exceptionexcept Exception:  log.warn("Failure committing transaction, rolling back")  db.rollback()else:  log.info("Saved the new FOO")finally:  db.close()

In fact, we have added code at both ends. First, let's take a look at else. When no exception occurs, the code here will be executed. In our example, only information about successful transactions is written to the log, but you can perform more interesting operations as needed. One possible application is to start background tasks or notifications.

Obviously, the function of the finally clause here is to ensure that db. close () can always run. Looking back, we can see that all the Code related to data storage eventually forms a beautiful logical group at the same indentation level. When code maintenance is required in the future, we can intuitively see that these lines of code are used to complete the commit operation.
Context and Control

Previously, we have seen that exceptions are used to process the control flow. Generally, the basic steps are as follows:

• Try to get resources (files, network connections, etc)
• If it fails, clear all the items left
• If the resource is obtained successfully, perform the corresponding operation.
• Write logs
• Program ended

With this in mind, let's take a look at the database example in the previous chapter. We use try-submit T-finally to ensure that any transaction we start is either committed or rolled back.
 

try:  # attempt to acquire a resource  db.commit()except Exception:  # If it fails, clean up anything left behind  log.warn("Failure committing transaction, rolling back")  db.rollback()else:  # If it works, perform actions  # In this case, we just log success  log.info("Saved the new FOO")finally:  # Clean up  db.close()# Program complete

The previous example maps to the steps just mentioned. Does this logic change much? Not many.

We will perform the same steps almost every time we store data. We can write these logics into a method, or we can use context manager)
 

db = db_library.connect("fakesql://")# as a functioncommit_or_rollback(db) # context managerwith transaction("fakesql://") as db:  # retrieve data here  # modify data here

The context manager protects the code segment by setting the resources (Context Environment) required for the code segment to run. In our example, if we need to process a database transaction, the process will be as follows:

• Connect to database
• Start operation at the beginning of the code segment
• Submit or roll back at the end of the code segment
• Clear resources at the end of the code snippet

Let's create a context manager and use the context manager to hide the settings of the database. The contextmanager interface is very simple. The context manager object needs to have a _ enter _ () method to set the required context, and also needs a _ exit _ (exc_type, exc_val, exc_tb) the method is called after the code segment is left. If no exception occurs, all three exc _ * parameters are None.

The _ enter _ method here is very simple. Let's start with this function.
 

class DatabaseTransaction(object):  def __init__(self, connection_info):    self.conn = db_library.connect(connection_info)   def __enter__(self):    return self.conn

The _ enter _ method only returns the database connection. In the code segment, we use this database connection to access data. The database connection is actually established in the _ init _ method. Therefore, if the database fails to establish a connection, the code segment will not be executed.

Now let's define how the transaction will be completed in the _ exit _ method. There is a lot of work to be done here, because it is necessary to handle all exceptions in the code segment and to close the transaction.
 

def __exit__(self, exc_type, exc_val, exc_tb):    if exc_type is not None:      self.conn.rollback()     try:      self.conn.commit()    except Exception:      self.conn.rollback()    finally:      self.conn.close()

Now we can use the DatabaseTransaction class as the context manager in our example. Inside the class, the _ enter _ and _ exit _ Methods start to set up data connections and handle the aftermath.
 

# context managerwith DatabaseTransaction("fakesql://") as db:  # retrieve data here  # modify data here

To improve our (simple) Transaction Manager, we can add various exception handling methods. Even now, this transaction manager has hidden a lot of complicated processing for us, so you don't have to worry about database-related details every time you pull data from the database.
Generator

Generators introduced in Python 2 is a simple method to implement iteration. This method does not generate all values at once. A typical function action in Python is to start execution, perform some operations, and finally return the result (or not ).

The behavior of the generator is not like this.
 

def my_generator(v):  yield 'first ' + v  yield 'second ' + v  yield 'third ' + v print(my_generator('thing'))# <generator object my_generator at 0x....>

Replace return with the yield keyword, which is unique in the generator. When we call my_generator ('thing '), What I get is not the result of the function but a generator object, this generator object can be used in any place where we use a list or other iteratable objects.

A more common usage is to use the generator as part of a loop as in the following example. The loop continues until the generator stops the yield value.
 

for value in my_generator('thing'):  print value # first thing# second thing# third thing gen = my_generator('thing')next(gen)# 'first thing'next(gen)# 'second thing'next(gen)# 'third thing'next(gen)# raises StopIteration exception

After the generator is instantiated, it does not do anything until it is required to generate a value. At this time, it will always execute until the first yield is encountered and return this value to the caller, the generator then saves the context and suspends until the caller needs the next value.

Now we can write a generator that is more useful than the three hardcoded values returned just now. The typical generator example is an infinite Fibonacci series generator. Let's give it a try. The number of columns starts from 1 and returns the sum of the first two numbers in sequence.
 

def fib_generator():  a = 0  b = 1  while True:    yield a    a, b = b, a + b

The while True loop in the function should be avoided in general cases, because this will cause the function to fail to return, but it does not matter to the generator, as long as there is yield in the loop. When using this generator, we must add the end condition because the generator can continuously return values.

Now, we use our generator to calculate the first Fibonacci sequence value greater than 10000.
 

min = 10000for number in fib_generator():  if number > min:    print(number, "is the first fibonacci number over", min)    break

This is very simple. We can set any number to a large value, and the code will eventually generate the first value greater than X in the Fibonacci series.

Let's look at a more practical example. The paging interface is a common method to cope with application restrictions and avoid sending more than 50 MB of JSON data packets to mobile devices. First, we define the required API, And then we write a generator for it to hide the paging logic in our code.

The APIS we use come from Scream, a place where users discuss what they have or want to eat. Their search API is very simple, basically as follows.
 

GET http://scream-about-food.com/search?q=coffee{  "results": [    {"name": "Coffee Spot",     "screams": 99    },    {"name": "Corner Coffee",     "screams": 403    },    {"name": "Coffee Moose",     "screams": 31    },    {...}  ]  "more": true,  "_next": "http://scream-about-food.com/search?q=coffee?p=2"}

This generator will process paging and limit retry logic, which will work in the following logic:

• Receive the content to search
• Query the scream-about-food Interface
• Retry if the interface fails.
• One yield result at a time
• If yes, get the next page
• Exit if no more results exist.

Very simple. I will implement this generator. to simplify the code, we do not consider retrying the logic for the moment.
 

import requests api_url = "http://scream-about-food.com/search?q={term}" def infinite_search(term):  url = api_url.format(term)  while True:    data = requests.get(url).json()     for place in data['results']:      yield place     # end if we've gone through all the results    if not data['more']: break     url = data['_next']

When we create a generator, you only need to input the search content. Then the generator will generate a request and obtain the result if the result exists. Of course, there are some unhandled boundary problems. The exception is not handled. When the API fails or an unrecognized JSON is returned, the generator throws an exception.

Despite the incomplete processing, we can still use this code to obtain the sorting of our restaurant in the keyword "coffee" search results.
 

# pass a number to start at as the second argument if you don't want# zero-indexingfor number, result in enumerate(infinite_search("coffee"), 1):  if result['name'] == "The Coffee Stain":    print("Our restaurant, The Coffee Stain is number ", number)    returnprint("Our restaurant, The Coffee Stain didnt't show up at all! :(")

If Python 3 is used, you can also use the generator when using the standard library. When a function like dict. items () is called, the generator is returned instead of the list. To obtain this behavior, Python 2 adds the dict. iteritems () function, but it is rarely used.
Python 2 and 3 compatibility

Migrating from Python 2 to Python 3 is a tough task for any code library (or developer), but writing code that can run both versions is also possible. Python 2.7 will be supported by 2020, but many new features will not support backward compatibility. Currently, if you cannot completely discard Python 2, you 'd better use Python 2.7 and 3 + compatible features.

For comprehensive guidance on the features supported by the two versions, you can view the Porting Python 2 Code on python.org.

Let's take a look at the most common situation you will encounter when writing compatible code, and how to use _ ure _ as a work und.
Print or print ()

Almost every developer who switches from Python 2 to Python 3 will write an incorrect print expression. Fortunately, you can import the print_function module and use print as a function rather than a keyword to write a compatible print.
 

for result in infinite_search("coffee"):  if result['name'] == "The Coffee Stain":    print("Our restaurant, The Coffee Stain is number ", result['number'])    returnprint("Our restaurant, The Coffee Stain didn't show up at all! :(")Divided Over Division

From Python 2 to Python 3, the default division behavior also changes. In Python 2, the division of integers is only divided, and the fractional part is truncated. Most users do not want this behavior, so in Python 3, even the division between integers is also executed.
 

print "hello" # Python 2print("hello") # Python 3 from __future__ import print_functionprint("hello") # Python 2print("hello") # Python 3

This kind of behavior changes will lead to a series of minor bugs when writing code that runs on Python 2 and Python 3 at the same time. We need the _ ure _ module again. Importing division will produce the same running result for the code in both versions.
 

print(1 / 3) # Python 2# 0print(1 / 3) # Python 3# 0.3333333333333333print(1 // 3) # Python 3# 0