A summary of some pitfalls and tricks in Python

Python is a powerful language that is widely used, allowing us to dive into this language and learn some of the tricks of control statements, the tips of the standard library, and some common pitfalls.

Python (and its various libraries) is very large. It is used for system automation, Web applications, big data, data analysis, and security software. This document is designed to showcase a few tips that will lead you to a faster development, easier and more fun way to debug.

Like learning python and learning all other languages, the really useful resources are not the cumbersome, official documents of each language, but the ability to share knowledge using common grammars, libraries, and the Python community.
Explore standard data types
The humble Enumerate

Traversal is very simple in Python, using "for Foo in bar:" is OK.

drinks = ["Coffee", "tea", "milk", "water"]for drink in Drinks:  print (' Thirsty for ', drink) #thirsty for coffee#thirsty For tea#thirsty for Milk#thirsty for water

But using both the ordinal and the element itself is a common requirement. We often see some programmers using Len () and range () to iterate through the list by subscript, but there is an easier way.

drinks = ["Coffee", "tea", "milk", "water"]for index, drink in enumerate (drinks):  print ("Item {} is {}". Format (Index, Drink)) #Item 0 is Coffee#item 1 are tea#item 2 is Milk#item 3 is water

The enumerate function can traverse elements and their ordinal numbers at the same time.
Set type

Many concepts can be attributed to the operation of collections (set). For example: Confirm that a list has no duplicate elements, see two elements in a list, and so on. Python provides a set data type to make operations like this faster and more readable.

# Deduplicate A list *fast*print (set (["Ham", "eggs", "bacon", "Ham"])) # {' Bacon ', ' eggs ', ' ham '} # Compare lists to find D ifferences/similarities# {} without "key": "Value" pairs makes a setmenu = {"Pancakes", "ham", "eggs", "bacon"}new_menu = { "Coffee", "ham", "eggs", "bacon", "bagels"} new_items = New_menu.difference (menu) print ("Try Our new", ",". Join (New_items ) # Try Our new bagels, coffee Discontinued_items = menu.difference (new_menu) print ("Sorry, we no longer has", ",". Join (d  Iscontinued_items) # Sorry, we no longer has pancakes Old_items = new_menu.intersection (menu) print ("Or get the same old", ",". Join (Old_items)) # or get the same old eggs, bacon, ham full_menu = new_menu.union (menu) print (' At one time ' or another , we ' ve served: ",", ". Join (Full_menu)) # At one time or another, we ' ve served:coffee, ham, pancakes, bagels, bacon, eggs

The intersection function compares all the elements in the list, returning the intersection of two sets. In our example, the staple food for breakfast is bacon, eggs and ham.
Collections.namedtuple

If you don't want to add a method to a class, but want to use Foo.prop's calling method, then you need namedtuple. You define the class attributes in advance, and then you can instantiate a lightweight class that consumes less memory than the complete object.

Lightobject = namedtuple (' Lightobject ', [' shortname ', ' Otherprop ']) m = lightobject () m.shortname = ' athing ' > Traceback (most recent call last):> Attributeerror:can ' t set attribute

In this way you cannot set the properties of the Namedtuple, as you cannot modify the values of the elements in a tuple (tuple). You need to set the value of the property when instantiating the Namedtuple.

Lightobject = namedtuple (' Lightobject ', [' shortname ', ' Otherprop ']) n = lightobject (shortname= ' Something ', otherprop= ' Something Else ') N.shortname # somethingcollections.defaultdict

When writing a python application using a dictionary, there are many times when some keywords do not exist at first, such as the following example.

Login_times = {}for T in logins:  if Login_times.get (T.username, None):    login_times[t.username].append ( T.datetime)  else:    login_times[t.username] = [T.datetime]

Using Defaultdict We can skip the logic of checking the presence of a keyword, and any access to an undefined key will return an empty list (or other data type).

Login_times = collections.defaultdict (list) for T in logins:  login_times[t.username].append (T.datetime)

You can even use a custom class to instantiate a class when called.

From datetime import Datetimeclass Event (object):  def __init__ (self, t=none):  if T is None:    self.time = Datet Ime.now ()  else:    self.time = t events = collections.defaultdict (Event) for E in user_events:  print (events[ E.name].time)

If you want to have defaultdict features, and you want to handle nested keys in a way that accesses properties, then you can look at addict.

Normal_dict = {' A ': {' B ': {' c ': {'    d ': {          ' e ': ' Really really nested dict '  }}} } from addict Import dictaddicted = Dict () addicted.a.b.c.d.e = ' really really nested ' print (addicted) # {' A ': {' B ': {' c ': {' d ': {' E ': ' Really really nested '}}}}

This little program is much easier to write than the standard Dict. So why not defaultdict it? It looks simple enough, too.

From collections Import Defaultdictdefault = Defaultdict (dict) default[' A ' [' B '] [' C '] [' d '] [' e '] = ' really really nested Dict ' # fails

This code looks fine, but it eventually throws a Keyerror exception. This is because default[' a ' is dict, not defaultdict. Let's construct a value that is dictionaries of the defaulted defaultdict type, which can only resolve level two nesting.

If you just need a default counter, you can use collection. Counter, this class provides a number of handy functions, such as Most_common.
Control flow

When learning about control structures in Python, it is often necessary to study for, While,if-elif-else, and try-except carefully. As long as it is used correctly, these control structures can handle most situations. It is also for this reason that almost all of the languages you encounter provide similar control structure statements. In addition to the basic control structure, Python also provides some of the less commonly used control structures that make your code more readable and maintainable.
Great Exceptations

Exceptions, as a control structure, is very common when working with databases, sockets, files, or any resource that might fail. Writing database operations using standard try, except structures is usually a way of typing.

Try:  # get API data  data = Db.find (id= ' foo ') # may raise exception  # manipulate the data  db.add (data)  # Save it Again  Db.commit () # may raise exceptionexcept Exception:  # Log the failure  db.rollback () db.close ()

Can you find the problem here? There are two possible exceptions that will trigger the same except module. This means that finding data failure (or failing to establish a connection to the query data) causes a fallback operation. This is definitely not what we want, because at this point in time the transaction does not begin. The same fallback should not be the correct response to the database connection failure, so let's deal with different situations separately.

First, we will process the query data.

Try:  # get API data  data = Db.find (id= ' foo ') # may raise exceptionexcept Exception:  # Log the failure and bail  out Log.warn ("Could not retrieve FOO")  return # manipulate the Datadb.add (data)

Data retrieval now has its own try-except so that when we don't get the data, we can take any approach. Without the data our code is unlikely to do anything useful anymore, so we'll just exit the function. In addition to exiting you can also construct a default object, retrieve it again, or end the entire program.

Now let's wrap up the commit code separately so that it can be more elegant and error-handling.

Try:  Db.commit () # Raise exceptionexcept Exception:  log.warn ("Failure committing transaction, rolling back" )  Db.rollback () Else:  log.info ("Saved The New FOO") Finally:  Db.close ()

In fact, we've added both ends of the code. First, let's look at the else, which executes the code here when no exception occurs. In our case, this is just a matter of writing the information about the success of the transaction to the log, but you can do more interesting things as needed. One possible application is to start a background task or notification.

It is clear that the function of the finally clause here is to ensure that db.close () is always able to run. Looking back, we can see that all the code related to data storage eventually forms a pretty logical grouping at the same indentation level. When code maintenance is required later, it is straightforward to see that these lines of code are used to complete the commit operation.
Context and Control

Previously, we have seen the use of exceptions to handle the flow of control. Typically, the basic steps are as follows:

• Try to get resources (files, network connections, etc.)
• If it fails, clear everything left behind.
• Successful access to resources is done accordingly
• Write log
• End of program

With that in mind, let's take a look at the example of the previous chapter of the database. We use try-except-finally to ensure that any transactions we start are either submitted or rolled back.

Try:  # attempt to acquire a resource  db.commit () except Exception:  # If It fails, clean up anything left behind  Log.warn ("Failure committing transaction, rolling back")  Db.rollback () Else:  # If It works, perform actions  # In this case, we just log success  Log.info ("Saved The New FOO") Finally:  # clean up  db.close () # Progra M complete

Our previous example maps almost exactly to the steps just mentioned. Does this logic change a lot? Not much.

Almost every time we store data, we'll do the same thing. We can write these logic into a method, or we can use the context Manager

db = Db_library.connect ("fakesql://") # as a functioncommit_or_rollback (db) # Context Managerwith transaction ("fakesql:/ /") as DB:  # Retrieve data here  # Modify data here

The context Manager protects the code snippet by setting the resource (context) that is required to run the code snippet. In our case, we need to deal with a database transaction, so the process will be:

• Connecting to a database
• Start at the beginning of the code snippet
• Commit or rollback at the end of a code snippet
• Clean up resources at the end of a code snippet

Let's create a context manager that uses the context Manager to work with the settings for our hidden database. The ContextManager interface is very simple. The context Manager object needs to have a __enter__ () method to set the desired context, and a __exit__ (Exc_type, Exc_val, Exc_tb) method is called after it leaves the code snippet. If there is no exception, then all three exc_* parameters will be none.

The __enter__ method here is very simple, let's start with this function.

Class Databasetransaction (object):  def __init__ (self, connection_info):    self.conn = Db_library.connect ( Connection_info)   def __enter__ (self):    return Self.conn

The __enter__ method simply returns the database connection, in which we use the database connection to access the data. The database connection is actually established in the __init__ method, so if the database fails to establish a connection, the code snippet will not execute.

Now let's define how the transaction will be done in the __exit__ method. There's a lot more work to do here, because all the exceptions in the code snippet are handled, and the transaction is closed.

def __exit__ (self, exc_type, Exc_val, EXC_TB):    If exc_type are not None:      self.conn.rollback ()     try:      Self.conn.commit ()    except Exception:      self.conn.rollback ()    finally:      self.conn.close ()

Now we can use the Databasetransaction class as the context manager in our example. Inside the class, the __enter__ and __exit__ methods will start and set up the data connection and handle the aftermath work.

# context Managerwith databasetransaction ("fakesql://") as DB:  # Retrieve data here  # Modify data here

To improve our (simple) transaction manager, we can add a variety of exception handling. Even now, this transaction manager has hidden a lot of complex processing for us, so you don't have to worry about database-related details every time you pull data from a database.
Generator

The builder introduced in Python 2 (generators) is an easy way to implement iterations that do not produce all the values at once. The typical function behavior in Python is to start execution, then take some action, and finally return the result (or not).

This is not the behavior of the generator.

def my_generator (v):  yield ' first ' + V  yield ' second ' + v  yield ' third ' + V print (my_generator (' thing ')) # 
 
  

Use the yield keyword instead of return, which is where the generator is unique. When we call My_generator (' thing '), I get not the result of a function but a generator object, which can be used anywhere we use lists or other objects that can iterate.

A more common use is to use the generator as part of the loop as in the following example. The loop will continue until the generator stops the yield value.

For value in My_generator (' thing '):  print Value # First thing# second thing# third thing gen = My_generator (' thing ') ne XT (Gen) # ' first thing ' next (gen) # ' second thing ' next (gen) # ' third thing ' next (gen) # raises stopiteration exception

After the generator is instantiated, it does nothing until it is required to produce a value, and then it executes until it encounters the first yield and returns the value to the caller, and then the generator holds the context and suspends until the caller needs the next value.

Now let's write a generator that is useful compared to the three hard-coded values that we just returned. The classic generator example is an infinite Fibonacci sequence generator, let's try it out. The sequence starts at 1 and returns the sum of the first two numbers in turn.

Def fib_generator ():  a = 0  b = 1  while True:    yield a    , B = B, A + b

The while True loop in a function should normally be avoided, as this causes the function to fail to return, but it does not matter to the generator, as long as the yield is guaranteed in the loop. When we use this generator, we should pay attention to the addition of the end condition, because the generator can continuously return the value.

Now, use our generator to calculate the first Fibonacci sequence value greater than 10000.

Min = 10000for number in Fib_generator ():  If number > min:    print (number, "was the first Fibonacci number over", min) Break    

This is very simple, we can set the value arbitrarily large, the code will eventually produce the Fibonacci sequence of the first value greater than X.

Let's look at a more practical example. The page turn interface is a common way to address application limitations and avoid sending a JSON packet larger than 50 megabytes to a mobile device. First, we define the required API, and then we write a generator for it to hide the paging logic in our code.

The API we use comes from scream, a place where users discuss restaurants they've eaten or want to eat. Their search API is very simple, basically the following.

GET http://scream-about-food.com/search?q=coffee{  "Results": [    {"name": "Coffee Spot",     "screams": 99    },    {"name": "Corner Coffee",     "screams": 403    },    {"name": "Coffee Moose",     "screams"    :-} ,    {...}  ]  " More ": True,  " _next ":" Http://scream-about-food.com/search?q=coffee?p=2 "}

They embed a link to the next page in the API answer, so it's easy to get to the next page. We can just get the first page without considering the page number. To get the data, we'll use a common requests library and encapsulate it with a generator to showcase our search results.

This generator will process paging and limit retry logic, which will work according to the following logic:

• Get what you're searching for
• Querying the Scream-about-food interface
• Retry if the interface fails
• Yield one result at a time
• If you have one, get the next page.
• When there are no more results, exit

Very simple. Let me implement this generator, in order to simplify the code we do not consider retry logic for the time being.

Import Requests Api_url = "Http://scream-about-food.com/search?q={term}" def infinite_search (term):  URL = api_ Url.format (term) and  True:    data = requests.get (URL). JSON () for place in     data[' results ']:      yield Place     # End If we ' ve gone through all the results    if not data[' more ': break     url = data[' _next ']

When we create the generator, you only need to pass in the content of the search, and then the generator will generate the request and get the result if the result exists. There are, of course, some unresolved boundary problems. The exception is not handled, and the generator throws an exception when the API fails or the unrecognized JSON is returned.

In spite of these unresolved areas, we can still use this code to get our restaurants sorted in the keyword "coffee" search results.

# Pass a number to start at as the second argument if you don ' t want# zero-indexingfor number, result in enumerate (infinit  E_search ("Coffee"), 1):  if result[' name '] = = "The Coffee Stain":    print ("Our restaurant, the coffee stain is number ", number)    Returnprint (" Our restaurant, the Coffee stain didnt ' t show up at all!:(")

If you use Python 3, you can also use the generator when you use the standard library. When a function like Dict.items () is called, the generator is returned instead of the list. In Python 2, in order to get this behavior, Python 2 adds the Dict.iteritems () function, but with less.
Python 2 and 3 compatibility

Migrating from Python 2 to Python 3 is a daunting task for any code base (or developer), but it is also possible to write code that works in two versions. Python 2.7 will be supported for 2020, but many new features will not support backwards compatibility. For now, if you're not yet completely abandoning Python 2, it's best to use Python 2.7 and the compatibility features.

For a comprehensive guide to the two version support features, you can see Porting Python 2 Code on python.org.

Let's take a look at the most common scenarios you'll encounter when you're planning to write compatible code, and how to use __future__ as a workaround.
Print or print ()

Almost every developer who switches from Python 2 to Python 3 writes out the wrong print expression. Fortunately, you can write a compatible print using print as a function instead of a keyword by importing the print_function module.

For result in Infinite_search ("Coffee"):  if result[' name '] = = "The Coffee Stain":    print ("Our restaurant, the Coff EE stain is number ", result[' number '])    returnprint (" We restaurant, the Coffee stain didn ' t show up at all!:(") Divided over Division

From Python 2 to Python 3, the default behavior of division also changed. In Python 2, the division of integers is only divisible, and the fractional parts are all truncated. Most users do not want this behavior, so even a division between integers in Python 3 performs a floating-point addition.

print "Hello" # python 2print ("Hello") # Python 3 from __future__ import print_functionprint ("Hello") # python 2print ("Hel Lo ") # Python 3

This change in behavior leads to a series of small bugs when writing code that runs on both Python 2 and Python 3. Once again we need the __future__ module. Importing division will cause the code to produce the same results in two versions.

Print (1/3) # python 0print (1/3) # python 3# 0.3333333333333333print (1//3) # python 3# 0

