On the joint use of Python program and C + + program _python

As a Python programmer, you should be able to face up to Python's pros and cons. Python is slow to run, especially when it comes to large amounts of data, and Python is slow to endure. In this case, the "professional" solution is to use NumPy or OpenCL. But sometimes it's not cost-effective to use this heavy solution for a little bit of functionality, or sometimes the actions you want to implement are not in NumPy, and we need to write them in C. In short, we use Python and C + + mixed programming to speed up the operation of program hotspots.

The first thing to note is that before you consider joint programming, be sure to find a hot spot for the program to run. In a nutshell, using the standard library profile or Cprofile module to find the most CPU-consuming location, if this location is only a simple consumption of IO time, usually replaced by C + + program meaning will not be very large, at this time do joint programming may be ineffective, do not have much effect.

There are also situations where Python programmers want to use the functions provided by the operating system or external modules. These modules are typically provided for C/s + + programmers. This is where Python and C + + programming are joined.

The Python language can be said to be the best glue language. Only in conjunction with C + + programming, the Python community offers the following solutions, depending on the difficulty and functionality of using them:

1. Use the standard library ctypes to directly invoke the dynamic link library written by C + +. This is the easiest and most easy-to-use solution. C + + programmers use their own rich experience to implement the intended function as a dynamic link library. Python programmers can simply invoke the name, parameter type, and return value type of these dynamic-link library functions. When you pass in the parameter, the cTYPES module automatically makes the object of Python the type of parameter that is required for C + +. For example, the following API that invokes Windows:

  #定义参数类型与函数名称
  from ctypes.wintypes import UINT, DWORD
  getlastinputinfo = Ctypes.windll.user32.GetLastInputInfo
  class Lastinputinfo (ctypes. Structure):
    _fields_ = [("cbsize", UINT),
         ("Dwtime", DWORD)]

  #开始调用DLL导出的函数
  def getlastinputtime_nt ( ):
    info = lastinputinfo ()
    info.cbsize = ctypes.sizeof (info)
    info.dwtime = 0
    if not getlastinputinfo ( Ctypes.byref (Info)):
      Raise Windowserror ("") return
    Info.dwtime

Here's how to construct the structure needed for the Windows API, how to populate the structure, and analyze the return value.

cTYPES can also provide a Python function to C + + code as a callback function.

Compared with other solutions. cTYPES does not require programmers to be familiar with the C + + language, and does not need to install a C + + compiler, which operates directly through the interface of the operating system. And cTYPES is part of the standard library and can be used directly if you have Python installed. These several reasons make it popular with Python programmers.

And what is its disadvantage. First, ctypes cannot simply invoke C + + programs, because C + + uses the name mangling technique to implement the overload of a function at compile time. C + + automatically adds the class name prefix to the member functions of the class. Therefore, C + + programmers need to use the C language of the calling convention to provide interfaces, no classes, no overloaded functions, no templates, no C + + exceptions. The inability to call existing C + + code directly may be the biggest drawback to this scenario.

In addition, for data types such as list, set, cTYPES cannot recognize and automatically convert between Python and C + + data types. The C + + section does not recognize Python data types, and only the Python language can be used to write the conversion code. If you have large amounts of data, or if you call them very often, converting your code will waste a lot of resources. This may be one of the other disadvantages of ctypes.

2. If you are using Jython or IronPython, they also provide modules similar to cTYPES, which can directly access modules written in Java or. NET languages. Its advantages and disadvantages are roughly similar to those of ctypes. Because of its limited scope of use, this is no longer detailed.

3. Use the Cython language, a new language similar to the Python language to write the predefined functionality of the code, and then convert the code into a C language compiled into a Python language can be directly called binary modules. Cython language is a new language which combines Python language with C language. It understands the syntax of the Python language itself, and then adds some syntax for the C language to more granular control over the data types and pointers. The basic compatibility of Python syntax is the most common feature of this solution. Many times, Python programmers can quickly speed up the old Python program by simply declaring the parameters and types of variables used in the code in the old code.

Cython provides a tool called Pyximporter that generates the appropriate Python module for a simple Cython program on a computer that has a C + + compiler installed. This makes the use of Cython as simple as an ordinary Python program. For example, the following code, directly saved as Myhello.pyx can be invoked.

  #myhello. Pyx
  def sayhellotentimes ():
    cdef int i #只要简单地为变量标识类型即可加速循环.
    for I in range (0):
      print ("Hello, world!")

  $ python
  >>> import pyximport pyximport.install ()
  >>> import Myhello
  >>> Myhello.sayhellotentimes ()

This shows that Cython is very easy to use. And not only can handle C language modules, but also processing C + + modules-although there is no direct support for virtual functions such as the full C + + features. Since it does not directly use C + + syntax, it is designed to be more concise and elegant than the new syntax, and therefore has a great attraction for programmers unfamiliar with C + +. Because parameter type conversion is more intelligent and efficient than ctypes, it usually improves efficiency.

Disadvantage, the so-called use of Python programmers skilled grammar to write high-speed operation code, at first glance is quite attractive. But if you want to get more control over memory and data structures, programmers may find that now he has to master the C + + language skillfully and write it in Cython syntax. In the lazy character of programmers, this is a difficult thing to endure. This may be the main reason why Cython itself is not popular.

4. Use Boost.python. Interestingly, in stark contrast to Ctypes/cython, Boost.python tends to have a more familiar programming environment for C + + programmers. It allows C + + programmers to use his familiar C + + syntax to directly control Python's data structure and invoke the Python interpreter. Instead of inventing new syntax like Cython, it uses the syntax of C + + directly to write interfaces for Python to use. As with Cython, its efficiency is better than that of ctypes.

Programmers only need to learn the two languages of C + + and Python, compared to the CYTHON/SWIG/SIP schemes. In addition, it is ideal for controlling Python code in programs written primarily by C + +, compared to several of the solutions mentioned in this article. Not only more powerful, but also more efficient. What are the disadvantages of such an amazing solution? Some people may disagree with it, the old fish once heard that it relies on boost on the wither, feel the compilation and learning of large and strange boost very waste of life.

5. Using swig or SIP, by writing an interface file, using a similar to C + + syntax-declaring functions, types of information, and then using special tools to generate Python interface code for C + + code. These interface codes can transform data structures between Python and C + +. Eventually compile these interface codes and become Python's binary modules. The interface file for swig and sip is very similar to the header files of c/A + +.

These two kinds of tools are similar, because. Essentially, they are all similar to Cython, using an intermediate language to generate the transformation code. But Swig/sip's ability to embed C + + in their interface files allows programmers to carefully adjust the conversion process for data types. In use, it is lower than the Cython level and closer to the API provided by Python itself.

Swig can generate conversion code for a variety of scripting languages. SIP is specifically for Python and C + +. In addition, SIP itself is developed as a specialized tool for PYQT, so it can understand the signal/slot of QT. From the application project, Swig seems to be a little more extensive. And sip, currently seen in the project is basically related to PYQT. It is said that Swig for C + + support is not good, do not know if anyone to say it. In contrast, SIP support for C + + is very perfect, such as virtual functions, protected member function, template, destructor, exception and other features are well supported. And sip supports the Python Gil and has a compilation system written using Python. It might be a little more convenient.

However, after all, this scheme must learn a new language, so on the surface, it is not as good as Cython and Boost.python. When programmers want to carefully adjust the type conversion code, they need to learn the internal mechanism of SWIG/SIP and be limited to the use of special variable names. This makes the learning curve of this scheme relatively high.

6. The direct use of the Python API can be called the Final Solution. Cython, SWIG, the SIP's interface file conversion generated after the C + + code is actually using the Python API. Compared to other scenarios, this scenario is quite complex, you must write data conversion code for each function call, and also worry about the reference count of the Python object. I think this kind of scheme is worthless, then I will not say more. Other tools Pybindgen don't know what's going on. You can see if you are interested.

All right. By the way, I've always felt that cTYPES and Xmlrpc are two great artifacts of the Python language that best reflect the productivity of Python.

I hope this article can provide a little help when you choose a technical route.