On the joint use of Python program and C + + program

As a Python programmer, you should be able to look squarely at the pros and cons of Python. In all the weeks, Python is running slowly, especially when it comes to large data volumes, and Python is slow to tolerate. In this case, the "professional" solution is to use NumPy or OpenCL. But sometimes in order to use this heavy-duty solution for a small function is not cost-effective, or sometimes want to implement the operation in NumPy, we need to write in C language. In short, we use Python's mixed programming with C + + to speed up the operation of hot program hotspots.

First of all, it is important to note that before considering joint programming, it is important to find the hotspot where the program runs. Simply, using the standard library profile or Cprofile module to find the most CPU-consuming location, if this location is only simple consumption of IO time, usually replaced by C + + program meaning is not very large, at this time to do joint programming may be less effective, do not have much effect.

In some cases, Python programmers want to use the functions provided by the operating system or external modules. These modules are typically provided for C + + programmers. This is also where Python and C + + programming together.

The Python language can be said to be the best glue language. Just as with C + + co-programming, the Python community offers the following solutions, depending on the difficulty and functionality of the application:

1. Use the standard library ctypes to directly invoke a dynamic-link library written in C + +. This is the simplest and easiest to use solution. C + + programmers use their rich experience to implement scheduled functions as dynamic link libraries. Python programmers can simply invoke the name, parameter type, and return value type of these dynamic link library functions as long as they know it. When you pass in a parameter, the cTYPES module automatically makes the Python object the type of parameter that the C + + language corresponds to. For example, the following API calls Windows:

  #定义参数类型与函数名称 from  ctypes.wintypes import UINT, DWORD  getlastinputinfo = Ctypes.windll.user32.GetLastInputInfo  class Lastinputinfo (ctypes. Structure):    _fields_ = [("cbsize", UINT),         ("Dwtime", DWORD)]  #开始调用DLL导出的函数  def getlastinputtime_nt ( ):    info = lastinputinfo ()    info.cbsize = ctypes.sizeof (info)    info.dwtime = 0    if not getlastinputinfo ( Ctypes.byref (Info)):      Raise Windowserror ("")    return Info.dwtime

Here we show how to construct the structure required for the API of Windows, how to populate the structure and parse the return value.

cTYPES can also provide a Python function to the C + + code as a callback function.

Compared to other solutions. cTYPES does not require a programmer to be familiar with the C + + language, it does not require a C + + compiler, and it directly operates through the interface of the operating system. And cTYPES is part of the standard library and can be used directly if Python is installed. These reasons make it popular with Python programmers.

And it's a disadvantage. First, ctypes cannot simply invoke a C + + program, because C + + uses the technique of name mangling to overload the function at compile time. C + + automatically adds a class name prefix to the class's member functions. Therefore, C + + programmers need to use the C language calling convention to provide interfaces, no classes, no overloaded functions, no templates, no C + + exceptions. The inability to invoke existing C + + code directly may be the biggest drawback of this scenario.

In addition, for data types such as list, set, cTYPES does not recognize and automatically translates between Python and the C + + data type. The Python data type is not recognized by the C + + section, which can only be written in the Python language. If the amount of data is large, or the call is frequent, the conversion code will waste a lot of resources. This may be one of the other disadvantages of ctypes.

2. If you are using Jython or IronPython, they also provide a module similar to cTYPES, which provides direct access to modules written in Java or. NET language. Its advantages and disadvantages are roughly similar to ctypes. Because of its limited scope of use, it is no longer detailed here.

3. Use the Cython language, a new language similar to the Python language, to write the code for a predetermined function, and then convert the code into a C language that compiles into a binary module that the Python language can invoke directly. Cython language is a new language that blends Python language with C language. It itself understands the syntax of the Python language, and then adds some C-language syntax on top of it to give finer control over data types and pointers. The basic compatibility of Python syntax is the biggest feature of this solution. In many cases, Python programmers can quickly speed up the old Python program by simply declaring the parameters and variable types used in the code in the old code.

Cython provides a tool called Pyximporter that can generate a Python module directly for a simple Cython program on a computer that has a C + + compiler installed. This makes the use of Cython as simple as a normal Python program. For example, the following code, directly saved as Myhello.pyx can be called.

  #myhello. Pyx  def sayhellotentimes ():    cdef int i #只要简单地为变量标识类型即可加速循环. For    i in range (0, ten):      print ("Hello, world!")  $ python  >>> import pyximport; Pyximport.install ()  >>> import Myhello  >>> Myhello.sayhellotentimes ()

This shows that Cython is very easy to use. And not only can handle the C language module, but also can handle C + + modules-although there is no direct support for virtual functions such as the complete C + + features. Because it does not use the C + + syntax directly, but also designs a new syntax that is more concise and elegant than that of C + +, it is attractive to programmers who are unfamiliar with C + +. Because parameter type conversions are more intelligent and efficient than ctypes, they can often improve efficiency.

Disadvantage, the so-called Python programmer skilled Grammar to write high-speed operation code, at first blush quite attractive. But if you want to have more in-depth control over memory and data structures, the programmer may find that now he has to master the C + + language skillfully and write it in Cython syntax. With the lazy character of programmers, this is an intolerable event. This may be the main reason why the Cython itself is not very popular.

4. Use Boost.python. Interestingly, in stark contrast to Ctypes/cython, Boost.python tends to have a more familiar programming environment for C + + programmers. It allows C + + programmers to use the familiar C + + syntax to directly control Python's data structure and invoke the Python interpreter. Instead of inventing a new syntax like Cython, it uses the syntax of C + + to write an interface for Python to use. In the same way as Cython, it is more efficient than ctypes.

Programmers only need to learn the two languages of C + + and Python compared to programs such as CYTHON/SWIG/SIP. In addition, it is ideal for controlling Python code in programs written primarily by C + +, compared to several of the solutions mentioned in this article. It is more powerful and more efficient. What are the disadvantages of such a magical solution? Some people may disagree, the old fish, as soon as they hear that it relies on boost, they feel like compiling and learning the huge and strange boost is a waste of life.

5. Using swig or SIP, by writing an interface file, using a C + + syntax similar to the one that declares functions, types, and then uses special tools to generate Python interface code for code C + +. These interface codes are able to transform the data structure between Python and C + +. Finally, these interface codes are compiled to become Python binary modules. The interface file for the swig and sip is very similar to the header file for C + +.

These two tools are similar, because. Essentially, they are all similar to Cython and use intermediate languages to generate the conversion code. But Swig/sip is able to embed C + + in their interface files, allowing programmers to carefully adjust the conversion process for data types. In use, it is lower than the Cython level and closer to the API provided by Python itself.

Swig can generate conversion code for a variety of scripting languages. SIP is specifically designed for Python and C + +. In addition, SIP itself is developed as a specialized tool for PYQT, so it is able to understand Qt's signal/slot. From the application project, Swig seems to be a little broader. SIP, currently seen in the project is basically related to PYQT. It is said that Swig for C + + support is not good, do not know if anyone to say it. In contrast, SIP is very well supported for C + +, such as virtual functions, protected member functions, templates, destructors, exceptions, and so on. And sip supports Python's Gil and has a compiled system written using Python. Might be a little more convenient.

However, after all, this kind of program must learn a new language, so on the surface, Cython and Boost.python. When programmers want to carefully adjust the type conversion code, they need to learn the internal mechanism of SWIG/SIP, which is limited to using special variable names. This makes the learning curve of this scheme relatively high.

6. Using the Python API directly can be called the Final Solution. Cython, SWIG, SIP's interface files are generated after the conversion of C + + code that actually uses the Python API. Compared to other scenarios, this scenario is quite cumbersome, and you must write the Data transformation code for each function call and also worry about the reference count of the Python object. I feel that this kind of scheme is useless, and then I will not talk about it any more. Other tools Pybindgen do not know what the situation is. If you are interested, you can take a look.

All right. On the other side, I have always felt that cTYPES and xmlrpc two great artifacts of Python language, which best embody the productivity of Python.

I hope this article will provide a little help when you choose a technical route.

