Counting methods to improve the efficiency of Python code _python

The first trick: Snake hit seven inches: Positioning bottlenecks

First, the first step is to locate bottlenecks. A simple chestnut, a function can be optimized from 1 seconds to 0.9 seconds, another function can be optimized from 1 minutes to 30 seconds, if the cost is the same, and the time limit can only get one, Which? According to the principle of the short plate, of course, choose the second one.

An experienced programmer would hesitate here, and so on? Function? So, also consider the number of calls? If the first function needs to be called 100,000 times throughout the program, and the second function is called 1 times throughout the program, this is not necessarily the case. This chestnut, is to show that the bottleneck of the program sometimes not necessarily a glance can be seen. Or the top choice, you should have a sense of the programmer, in most cases: a function that can be optimized from one minute to 30 seconds is more likely to capture our attention than a function that can be optimized from 1 seconds to 0.9 seconds, because there is a lot of room for improvement.

So, with so much nonsense to finish, offer the first recruit, profile. This is Python's own location program bottleneck of the tool! Although it provides three options for Profile,cprofile,hotshot. Also divided into built-in and external. However, the individual feels a foot, external cprofile. The centroid is as follows:

Python-m Profile Tease ratio program. py

The effect of this trick is to output a series of things, such as how many times a function is called, how much time it takes, how many of these are the function's child functions, how many times each time, and so on. A picture wins thousand words:

Filename:lineno (function): File name: first few lines (function name)
Ncalls: This item has been called several times
Tottime: How much time did it take to get rid of the internal functions of the boys?
Percall: Average time spent per call, Tottime divided by ncalls
Cumtime: This one has all of its internal functions. The total cost of the boys
Percall: Almost as much as the top percall, but Cumtime divided by Ncalls.
Find the best point to optimize, and then do it.

The second trick: a snake Zen: just a trick

When I first started touching python, one of my seniors told me that Python had a great idea, and it wanted everyone with it to write exactly the same program. Python's Zen is a cloud:

There should be one--and preferably only one--obvious to do it

So the Python professional Zen master offers some of the most common features of only one. I looked at the legendary Pythonwiki:performancetips, summed up a few "do not purple" to purple.

Do not purple when merging strings:

s = "" for substring in list:s + = substring

to purple:

s = "". Join (Slist)

Do not purple when formatting strings:

out = " 
 

to purple:

out = '  
 

Do not use loops when you do not need to cycle, such as do not purple:

NewList = [] for word in Oldlist:newlist.append (Word.upper ())

to purple:

NewList = Map (str.upper, Oldlist)

or purple:

NewList = [S.upper () for s in Oldlist]

Dictionary initialization, more commonly used:

Wdict = {} for word in Words:if Word isn't in wdict:wdict[word] = 0 Wdict[word] = 1

If you have too many words to repeat, consider using a purple pattern to save a lot of judgment:

Wdict = {} for Word in Words:try:wdict[word] + = 1 except keyerror:wdict[word] = 1

Try to minimize the number of function calls and replace them with internal loops, such as not purple:

x = 0 def doit1 (i): Global x x = x + I list = range (100000) T = Time.time () for I in List:doit1 (i)

to purple:

x = 0 def doit2 (list): Global x for i in list:x = x + I list = range (100000) T = Time.time () doit2 (list)

The third trick: the snake Sniper: high-speed search

This part comes from Ibm:python Code performance optimization techniques, the highest level of search algorithm is O (1) algorithm complexity. Which is the Hash Table. I was fortunate to learn some data structure when I was a bachelor. Know that Python's list is implemented using a method similar to a linked list. If the list is very large, in the vast number of items inside with if X in list_a to do search and judge the efficiency is very low.

Python's tuple I use very little, do not comment. The other two I used very much are set and Dict. These two are the same way of implementing Hash Table.

So try not to purple:

K = [10,20,30,40,50,60,70,80,90] for I-xrange (10000): If I in K: #Do something continue

to purple:

"' k = [10,20,30,40,50,60,70,80,90] k_dict = {i:0 for i in k}

First, convert the list into dictionary.

For I in Xrange (10000): If I in k_dict: #Do something Continue ' "

Find the intersection of list, do not purple:

List_a = [1,2,3,4,5]
list_b = [4,5,6,7,8]
List_common = [A For a in-list_a if a in list_b]

to purple:

List_a = [1,2,3,4,5]
list_b = [4,5,6,7,8]
List_common = set (list_a) &set (list_b)

Four strokes: Snake snakes ... : Can't think of a name, is a variety of small tips

Variable exchange does not require intermediate variables: A,b = B,a (there is a pit of God, so far the memory is profound: True,false = false,true)
If the use of python2.x, with xrange instead of range, if the python3.x,range is already xrange, Xrange has been wood. Instead of generating a list like range, Xrange generates an iterator that saves memory.
You can use X>y>z instead of x>y and y>z. Higher efficiency and better readability. Theoretically x>y, of course.
Add (x,y) is generally faster than a+b? This I have doubts, the experiment, first add can not be used directly, to import operator, second, my experiment results show that add (X,y) completely did not a+b fast, let alone also sacrifice readability.
While 1 is indeed a little faster than while True. Two experiments were done, about 15% faster.
No snakes, no snakes: Performance outside the code

Outside of the code, in addition to the hardware, is the compiler, here is a grand recommendation PyPy. PyPy is an instant compiler called Just-in-time. The compiler is characterized by the compilation of a running sentence, and the difference between the static compiler, I see a very vivid metaphor:

Assuming you're a director, static compilation is a way for the actor to recite the entire script and then perform for one hours in a row. Dynamic compilation is to allow the actor to perform for two minutes, then think about it, then look at the script, and then perform two minutes ...

Dynamic compilation and static compilation have their own strengths, see you play a movie or a play.

In addition, there is a cython can be built in Python some C code. I use very little, but the critical moment is really effective.