Python Performance Improvement method

some of the scenarios for Python performance improvements.

One, function call optimization (space span, avoid access to memory)

The core point of optimization of the program is to minimize the operation span, including the span of code execution time and the space span in memory.

1. Big Data summation, using sum

A = range (100000)%timeit-n sum (a) loops, best of 3:3.15 ms per Loop%%timeit  ...: s = 0  ...: For i in a:
  
    ...:  s + = i ...  : loops, Best of 3:6.93 ms Per loop
  

2. Sum small data, avoid using sum

%timeit-n s = a + B + C + D + E + F + G + H + i + j + K # The data volume is small and is accumulated faster than loops, best of 3:571 ns per Loop%timeit -n + s = SUM ([a,b,c,d,e,f,g,h,i,j,k]) # Small Data volume call sum function, space efficiency reduced by loops, best of 3:669 ns per loop

Conclusion: The sum of big data is efficient and the sum of small data is high.

Second, for loop optimization elements (using stacks or registers, to avoid access to memory)

For LST in [(1, 2, 3), (4, 5, 6)]: # LST index requires extra overhead  pass

You should avoid using indexes as much as possible.

For a, B, C in [(1, 2, 3), (4, 5, 6)]: # Better  Pass

is equivalent to assigning a value directly to each element.

Def force (): LST = range (4) for A1 in [1, 2]: for   A2 in LST: for     A3 in LST: for B1 in       LST: for B2 in         lst:< C4/>for B3 in LST:             for C1 in LST: for               C2 in LST: For                 C3 in LST:                   for D1 in LST:                     yield (a1, A2, A3, B1, B2, B3, C1, C2, C3, D1)                      %%timeit-n 10for T in force ():  sum ([t[0], t[1], t[2], t[3], t[4], t[5], t[6], t[7], t[8 ], t[9]]) loops, best of 3:465 ms per loop%%timeit-n 10for A1, A2, A3, B1, B2, B3, C1, C2, C3, D1 in force ():  sum ([A1, A2, A3, B1, B2, B3, C1, C2, C3, D1]) loops, best of 3:360 ms Per loop

Third, generator optimization (check table instead of operation)

Def force (Start, end): # Used for password brute-cracking program for  I in range (Start, end): Now    = i    sublst = [] for    J in range: 
  sublst.append (i% 10) # division operation overhead, larger than multiplication      i//=    sublst.reverse ()    yield (tuple (sublst), now)

Def force (): # Better LST = range (5) for A1 in [1]: for   A2 in LST: for     A3 in LST: for B1 in       LST: for         B2 In LST: For           B3 in LST: for             C1 in LST: for C2 in LST: for C3 in                 lst: for D1 in                   LST:                     yield (A1 , A2, A3, B1, B2, B3, C1, C2, C3, D1)

R0 = [1, 2] # readability and flexibility R1 = Range (Ten) r2 = R3 = R4 = R5 = R6 = R7 = R8 = R9 = R1force = ((a0, A1, A2, A3, A4, A5, A6, A7, A8, A9) for      A0 in r0 for A1 in R1 for A2 in R2 for A3 in R3 for A4 in R4 for A5 in R5 for A6 in R6 for A7 in R7 for      a 8 in R8 for A9 in R9)

Power Operation Optimization (POW (x, y, z))

def isprime (n):  if n & 1 = = 0:    return False  k, q = FIND_KQ (n)  a = Randint (1, n-1)  if POW (A, q, n ) = = 1: # Better than using a * * q% n operation optimization multiple    return True  for J in Range (k):    If Pow (A, POW (2, j) * Q, N) = = n-1: # A * * (( 2 * * j) * Q)% n      return True  return False

Conclusion: POW (x, y, z) is better than x**y%z.

V. Optimization of Division Operations

In [1]: From random import getrandbits in [2]: x = getrandbits (4096) in [3]: y = getrandbits (2048) in [4]:%timeit-n 1000 0 Q, R = Pmod (x, y) 10000 loops, best of 3:10.7 us per loop in [5]:%timeit-n 10000 q, r = x//y, x% y10000 loops, best o F 3:21.2 US per loop

Conclusion: Pmod is better than//and%.

vi. optimization of the algorithm time complexity　

The time complexity of the algorithm has the greatest impact on the execution efficiency of the program, and in Python it is possible to select the appropriate data structure to optimize the time complexity, such as the time complexity of the list and set to find an element, respectively O (n) and O (1). Different scenarios have different optimization methods, in general, there are divided, branch bound, greedy dynamic planning and other ideas.

Vii. reasonable use of copy and Deepcopy

For objects such as data structures such as dict and lists, the direct assignment uses a reference method. In some cases, you need to copy the entire object, you can use copy and deepcopy in the copy package, the difference between the two functions is that the deepcopy is recursive replication. Different efficiency:

In [MAX]: Import copyIn [+]:%timeit-n copy.copy (a) loops, best of 3:606 ns per Loopin [+]:%timeit-n Copy.dee Pcopy (a) loops, best of 3:1.17 US per loop

The Timeit followed by-n indicates the number of runs, and the next two lines correspond to the output of two Timeit. This shows that the latter is one order of magnitude slower.

An example of copy:

>>> lists = [[]] * 3>>> lists[[], [], []]>>> lists[0].append (3) >>> lists[[3], [3], [3] ]

What happens is that [[]] is a list of only one element that contains an empty list, so all three elements of [[]] * 3 are (pointing to) this empty list. Modify any element of lists to modify this list. The efficiency of the modification is high.

Eight, using Dict or set to find elements

Python dictionaries and collections are implemented using hash tables (similar to the C + + standard library unordered_map), and the time complexity of finding elements is O (1).

In [1]: R = Range (10**7) in [2]: s = set (R) # occupies 588MB memory in [3]: D = dict ((i, 1) for I in R) # takes up 716MB memory in [4]:%timeit- N 10000 (10**7)-1 in r10000 loops, best of 3:291 ns per Loopin [5]:%timeit-n 10000 (10**7)-1 in s10000 loops, best of 3:121 NS per Loopin [6]:%timeit-n 10000 (10**7)-1 in d10000 loops, best of 3:111 ns per loop

Conclusion: The memory consumption of set is minimal and the dict run time is shortest.

Ix. Rational Use (generator) and yield (save memory)

In [1]:%timeit-n a = (I-I in range (10**7)) # Generator typically traverses more efficient loops, best of 3:933 ns per Loopin [2]:%timeit-n 10  A = [I for I in range (10**7)]10 loops, Best of 3:916 ms per Loopin [1]:%timeit-n for x in (I-I in range (10**7)): PASS10 loops, Best of 3:749 ms per Loopin [2]:%timeit-n-X in [I-I in range (10**7)]: Pass10 loops, Best of 3 : 1.05 s per loop

Conclusion: Use the generator to traverse as much as possible.

The above is some of the Python performance improvement programs, follow-up continue to supplement, need to see.