Python high-performance programming method 1

Python high-performance programming method 1 read Zen of Python and input import this in the Python parser. A novice Python may notice the word "resolution" and think that Python is just another scripting language. "It must be slow! "

There is no doubt that the Python program does not have a compilation language that is efficient and fast. even Python advocates will tell you that Python is not suitable for these fields. however, YouTube has used Python to provide services for tens of millions of video requests per hour. all you need to do is write efficient code and use external implementation (C/C ++) code when necessary. here are some suggestions to help you become a better Python Developer:

1. using built-in functions: you can use Python to write efficient code, but it is difficult to beat the built-in functions. verified. they are very fast. 2. use join () to connect to a string. you can use "+" to connect strings. however, because string is unchangeable in Python, each "+" operation creates a new string and copies the old content. A common usage is to use a single character modified by the Python array module. when the modification is complete, use the join () function to create the final string.

>>># This is good to glue a large number of strings

>>> For chunk in input ():

>>> My_string.join (chunk)

3. use Python to assign multiple values and exchange variables.

This is elegant and fast in Python:

>>> X, y = y, x

This is slow:

>>> Temp = x

>>> X = y

>>> Y = temp

4. use local variables whenever possible

Python retrieves local variables faster than global variables. This means that the "global" keyword is avoided.

5. try to use "in"

Use the "in" keyword. concise and fast.

>>> For key in sequence:

>>> Print "found"

6. use delayed loading acceleration

Import the "import" declaration to the function only when necessary. in other words, if some modules do not need to be used immediately, import them later. for example, you do not have to import a large number of modules as soon as you enable the acceleration program. this technology cannot improve overall performance. however, it helps you allocate the module loading time more evenly.

7. use "while 1" for an infinite loop"

Sometimes you need an infinite loop in the program. (for example, an instance that listens to sockets) although "while true" can accomplish the same thing, "while 1" is a single-step operation. this method can improve your Python performance.

>>> While 1:

>>># Do stuff, faster with while 1

>>> While True:

>>># Do stuff, slower with wile True

8. use list comprehension

In Python 2.0, you can use list comprehension to replace a large number of "for" and "while" blocks. using List comprehension is usually faster, and the Python parser can find in a loop that it is a predictable mode and optimized. the additional benefit is that list comprehension is more readable (functional programming) and can save an additional counting variable in most cases. For example, let's calculate the number of even numbers between 1 and 10:

>>> # The good way to iterate a range

>>> Evens = [I for I in range (10) if I % 2 = 0]

>>> [0, 2, 4, 6, 8]

>>> # The following is not so Pythonic

>>> I = 0

>>> Evens = []

>>> While I <10:

>>> If I % 2 = 0: evens. append (I)

>>> I + = 1

>>> [0, 2, 4, 6, 8]

9. use xrange () to process long sequences:

This saves you a lot of system memory, because xrange () generates only one integer element for each call in the sequence. In contrast, range () directly gives you a complete list of elements, which is unnecessary for loop.

10. use Python generator:

This can also save memory and improve performance. For example, a video stream can be sent in one byte block instead of the whole stream. For example,

>>> Chunk = (1000 * I for I in xrange (1000 ))

>>> Chunk

>>> Chunk. next ()

0

>>> Chunk. next ()

1000

>>> Chunk. next ()

2000

11. understand the itertools module:

This module is very effective for iteration and combination. Let's generate a list of all the permutation and combinations of [1, 2, 3]. only three lines of Python code are required:

>>> Import itertools

>>> Iter = itertools. permutations ([1, 2, 3])

>>> List (iter)

[(1, 2, 3), (1, 3, 2), (2, 1, 3), (2, 3, 1), (3, 1, 2), (3, 2, 1)]

12. sort the learning bisect module to keep the list:

This is a free tool for implementing binary search and fast insertion of ordered sequences. In other words, you can use:

>>> Import bisect

>>> Bisect. insort (list, element)

You have inserted an element into the list, and you do not need to call sort () again to maintain the sorting of containers, because this will be very expensive in long sequences.

13. Understanding the Python list is actually an array:

The list implementation in Python is not based on common single-chain tables in computer science that people usually talk about. The list in Python is an array. That is to say, you can use constant time O (1) to retrieve an element in the list, instead of searching from the beginning. What is the significance of this? When using the list object insert (), Python developers need to think twice. for example: >>> list. insert (0, item)

Inserting an element before the list is inefficient because all subsequent subscript in the list has to be changed. however, you can use list. append () effectively adds elements to the end of the list. select deque first, if you want to quickly insert or at two times. It is fast, because deque in Python is implemented using double-stranded tables. I will not talk about it more.

14. test members using dict and set: check whether an element exists in dicitonary or set, which is very fast in Python. This is because dict and set use hash tables. The search efficiency can reach O (1 ). Therefore, if you need to check the members frequently, use set or dict as your container.

>>> Mylist = ['A', 'B', 'C'] # Slower, check membership with list:

>>> 'C' in mylist

>>> True

>>> Myset = set (['A', 'B', 'C']) # Faster, check membership with set:

>>> 'C' in myset:

>>> True

15. use the sort () of the Schwartzian Transform ():

The native list. sort () function is very fast. Python sorts the list in the natural order. Sometimes, you need to sort in a non-natural order. For example, the IP address you want to sort by server location. Python supports custom comparison. you can use list. sort (CMP (), which is slower than list. sort () because it increases the overhead of function calls. If you have any questions about the performance, you can apply for Gutman-Rosler Transform based on Schwartzian Transform. it is only interested in the algorithms actually used. its brief working principle is that you can change the list and call the Python built-in list. sort ()-> faster without using list. sort (CMP ()-> slow.

16. cache results of the Python decorator:

The "@" symbol is the decoration syntax of Python. It is not only used for tracing, locking, or logging. You can describe a Python function and remember the call results for later use. This technology is called memoization. The following is an example:

>>> From functools import wraps

>>> Def memo (f ):

>>> Cache = {}

>>> @ Wraps (f)

>>> Def wrap (* arg ):

>>> If arg not in cache: cache ['Arg '] = f (* arg)

>>> Return cache ['Arg ']

>>> Return wrap

We can also use the decorator for the Fibonacci function:

>>> @ Memo

>>> Def fib (I ):

>>> If I <2: return 1

>>> Return fib (i-1) + fib (I-2)

The key idea here is: enhance the function (decoration) function, and remember each calculated Fibonacci value. if they are in the cache, you do not need to calculate them.

17. understand Python GIL (Global interpreter lock ):

GIL is necessary because the memory management of CPython is non-thread-safe. You cannot simply create multiple threads and want Python to run faster on multiple core machines. This is because GIL will prevent multiple native threads from simultaneously executing Python bytecode. In other words, GIL will serialize all your threads. However, you can use threads to manage multiple derivative process acceleration programs, which run independently outside your Python code.

18. familiar with Python source code like familiar with documents:

Some Python modules use C for performance. When performance is critical and official documentation is insufficient, you can freely explore the source code. You can find the underlying data structure and algorithms. The Python source code library is a great place: http://svn.python.org/view/python/trunk/Modules

Conclusion:

These cannot replace brain thinking. open the hood to fully understand the responsibilities of developers, so that they will not quickly piece together a spam design. the Python suggestions in this article can help you achieve good performance. if the speed is not fast enough, Python will need external forces to analyze and run external code. in the second part of this article.