The use of Python optimization techniques ctypes Improve execution speed _python

First, let's share a little pit that a person encounters when using Python's ctypes to invoke C library.

The problem here is a C function, and the return value is the malloc generated string address. Normal use also no problem, also used a period of time, did not find anything unusual.

This time in the test, found that the use of this process will appear "segment error", resulting in the program exit.

After troubleshooting, determine the problem is the C function return value problem, ctypes the default function return type is int type.

You need to set the return type in use, for example:

func.restype = c_char_p

Here we will discuss the use of the next cTYPES tips

The cTYPES library allows developers to develop using the C language. The introduction of the C language interface can help us do a lot of things, such as the need to invoke C code to improve performance of some small problems. It allows you to access the Kernel32.dll and Msvcrt.dll dynamic link libraries on Windows systems, as well as libc.so.6 libraries on Linux systems. Of course, you can also use your own compiled shared library

Let's take a look at a simple example, we use Python to find the prime number within 1000000, repeat the process 10 times, and calculate the elapsed time.

Import Math from
Timeit import Timeit


def check_prime (x):
  values = xrange (2, int (math.sqrt (x)) + 1) for
  I In values:
    if x i = = 0: Return
      False return
  True


def get_prime (n): Return
  [x for X in Xrange (2, N) If Check_prime (x)]

print Timeit (stmt= ' Get_prime (1000000) ', setup= ' from __main__ import get_prime ',
       number= 10)

Output

42.8259568214


Below, write a check_prime function in C, and then import it as a shared library (dynamic link library)

#include <stdio.h>
#include <math.h>
int check_prime (int a)
{
  int c;
  for (c = 2; c <= sqrt (a); C + +) {
    if (a%c = 0) return
      0;
  }
  return 1;
}

Use the following command to generate the. So (Shared object) file

gcc -shared -o prime.so -fPIC prime.c

Import ctypes
import math from
Timeit import timeit
Check_prime_in_c = ctypes. Cdll ('./prime.so '). Check_prime


def check_prime_in_py (x):
  values = xrange (2, int (math.sqrt (x)) + 1) for
  I In values:
    if x i = = 0: Return
      False return
  True


def get_prime_in_c (n): Return
  [x to X in Xrange ( 2, N) if Check_prime_in_c (x)]


def get_prime_in_py (n): Return
  [x for X in Xrange (2, N) if Check_prime_in_py (x)]
   py_time = Timeit (stmt= ' get_prime_in_py (1000000) ', setup= ' from __main__ import get_prime_in_py ',
         number=10)
c_time = Timeit (stmt= ' Get_prime_in_c (1000000) ', setup= ' from __main__ import Get_prime_in_c ',
        number=10)
print "Python version: {} seconds". Format (py_time)

print "C version: {} seconds". Format (C_time)

Output

Python version:43.4539749622 seconds
C version:8.56250786781 seconds

We can see the obvious performance gap there are more ways to determine whether a number is prime

Take a look at a complex point example Quick Sort

Mylib.c

#include <stdio.h> typedef struct _RANGE {int start, end;}

Range;
  Range New_range (int s, int e) {range r;
  R.start = s;
  R.end = e;
return R;
  } void Swap (int *x, int *y) {int t = *x;
  *x = *y;
*y = t;
  } void Quick_sort (int arr[], const int len) {if (len <= 0) return;
  Range R[len];
  int p = 0;
  r[p++] = New_range (0, len-1);
    while (p) {range = r[--p];
    if (Range.Start >= range.end) continue;
    int mid = Arr[range.end];
    int left = Range.Start, right = range.end-1;
      while (left < right) {while [Arr[left] < mid && left < right) left++;
      while (Arr[right] >= mid && left < right) right--;
    Swap (&arr[left], &arr[right]);
    } if (Arr[left] >= arr[range.end]) swap (&arr[left), &arr[range.end]);
    else left++;
    r[p++] = New_range (Range.Start, left-1);
  r[p++] = New_range (left + 1, range.end);

 }
}

gcc -shared -o mylib.so -fPIC mylib.c

One trouble with using cTYPES is that native C code uses types that may not correspond to Python's. Like here, what is an array in Python? List? is also an array in the array module. So we need to convert.

test.py

Import ctypes
Import time
import random

quick_sort = ctypes. Cdll ('./mylib.so '). Quick_sort
nums = [] for
_ in range (+):
  r = [Random.randrange (1, 100000000) for x in Xran GE (100000)]
  arr = (Ctypes.c_int * len (R)) (*r)
  nums.append ((arr, Len (r))

init = Time.clock () for
I in Range:
  quick_sort (nums[i][0], nums[i][1])
print "%s"% (Time.clock ()-init)

Output

1.874907

Compared to the sort method of the Python list

Import ctypes
Import time
import random

quick_sort = ctypes. Cdll ('./mylib.so '). Quick_sort
nums = [] for
_ in range (MB):
  nums.append ([Random.randrange (1, 100000000) For x in Xrange (100000)])

init = Time.clock () to
I in range (m):
  nums[i].sort ()
print "%s"% (Time.clo CK ()-init)

Output

2.501257


As for structs, you need to define a class that contains the corresponding fields and types

Class Point (ctypes. Structure):
  _fields_ = [(' x ', ctypes.c_double),
        (' y ', ctypes.c_double)]

In addition to importing our own written C language extension files, we can also directly import the library files provided by the system, such as Linux under the C standard library implementation glibc

Import time
import random from
ctypes import cdll
libc = Cdll. LoadLibrary (' libc.so.6 ') # Linux system
# libc = cdll.msvcrt # Windows system
init = Time.clock ()
randoms = [Random.ran Drange (1) for x in Xrange (1000000)]
print "Python version:%s seconds"% (Time.clock ()-init)
init = time.cl Ock ()
randoms = [(Libc.rand ()%) for x in Xrange (1000000)]
print "C version:%s seconds"% (Time.clock ()-I Nit

Output

Python version:0.850172 seconds
C version:0.27645 seconds

These are the basic skills of ctypes, for ordinary developers, basically enough

For more detailed instructions please refer to: http://docs.python.org/library/ctypes.html