Use C language to expand Python Functions

 

Pyton and C have their own advantages and disadvantages. python is fast and reliable, and many ready-made modules are available for use, but the execution speed is relatively slow. C language is just the opposite, its execution speed is fast, but its development efficiency is low. To make full use of the advantages of the two languages, it is better to use python to develop the entire software framework and use C to implement its key modules. This article describes how to use the C language to extend Python functions, and describes how to compile Python extension modules with examples.

Python is a powerful high-level scripting language. It is powerful not only in its own functions, but also in its excellent scalability, python has been favored by more and more people and has been successfully applied to the development of various large software systems.

Different from other common scripting languages, Python programmers can use APIs provided by the Python language to use C or C ++ for Functional extension of Python, in this way, you can use Python's convenient and flexible syntax and functions, and obtain almost the same execution performance as C or C ++. Slow execution speed is a common feature of almost all scripting languages, and it is also an important factor that has been criticized. Python cleverly solves this problem through the organic combination with the C language, therefore, the application scope of the script language is greatly extended.

When developing a real Software System Using python, C/C ++ is often used to expand python. The most common situation is that a library written in C already exists. You need to use some functions of this library in the Python language. In this case, you can use the extension functions provided by python. In addition, because python is essentially a scripting language, it is difficult for some functions to be implemented using python to meet the execution efficiency requirements of the actual software system, in this case, you can use the extension function provided by python to implement these key code segments in C or C ++ to provide program execution performance.

This article mainly introduces the C language extension interfaces provided by python, and how to use these interfaces and C/C ++ languages to perform Functional extension of Python, it also describes how to implement Python function expansion with specific examples.

 

 

 

Python is a scripting language implemented in C language. It has excellent openness and scalability, and provides convenient and flexible application interface (API ), this allows C/C ++ programmers to expand the functions of the python interpreter at various levels. Before using C/C ++ to Expand functions of Python, you must first understand the C language interface provided by the python interpretation.

Python is an object-oriented scripting language. All objects are expressed as pyobject In the python interpreter. The pyobject structure contains all member pointers of the python object, maintain the type information and reference count of Python objects. During Python extension programming, once Python objects must be processed in C or C ++, A pyobject structure should be maintained.

In Python C Language extension interfaces, most functions have one or more parameters of the pyobject pointer type, and most of the returned values are pyobject pointers.

To simplify memory management, Python implements the automatic garbage collection function through the reference counting mechanism. Each object in Python has a reference count, it is used to count the number of times the object is referenced in different places. Every time a python object is referenced, the corresponding reference count increases by 1. Every time a python object is destroyed, the corresponding reference is reduced by 1. Only when the reference count is zero, to delete Python objects from the memory.

The following example illustrates how the python interpreter manages pyhon objects by using reference counts:

Example 1: refcount. pyclass refcount: # etc. r1 = refcount () # reference count is 1r2 = R1 # reference count is 2del (R1) # reference count is 1del (R2) # reference count is 0, delete an object

 

When processing Python objects in C/C ++, correct maintenance of the reference count is a key issue. If it is not handled properly, memory leakage may occur. Python's C language interface provides some macros to maintain the reference count. The most common is to use py_incref () to increase the reference count of Python objects by 1, and use py_decref () to reduce the reference count of Python objects by 1.

Python defines six data types: integer, floating point, String, tuples, lists, and dictionaries, first, you need to know how to convert data types in C and python.

2.3.1 integer, floating point, and string

It is relatively simple to use integer, floating point, and string data types in Python C Language extension. You only need to know how to generate and maintain them. The following example shows how to use python in C:

Example 2: typeifs. C // build an integerpyobject * pint = py_buildvalue ("I", 2003); Assert (pyint_check (pint); int I = pyint_aslong (pint); py_decref (pint ); // build a floatpyobject * pfloat = compile ("F", 3.14f); Assert (pyfloat_check (pfloat); float F = pyfloat_asdouble (pfloat); py_decref (pfloat ); // build a stringpyobject * pstring = py_buildvalue ("S", "Python"); Assert (pystring_check (pstring); int nlen = pystring_size (pstring ); char * s = pystring_asstring (pstring); py_decref (pstring );

 

2.3.2 tuples

The tuples In the Python language are fixed-length arrays. When the python interpreter calls methods in the C language extension, all non-Keyword parameters are transmitted as tuples. The following example demonstrates how to use Python's tuples in C:

Example 3: typetuple. C // create the tuplepyobject * ptuple = pytuple_new (3); Assert (pytuple_check (ptuple); Assert (pytuple_size (ptuple) = 3 ); // set the values (ptuple, 0, py_buildvalue ("I", 2003); values (ptuple, 1, py_buildvalue ("F", 3.14f); pytuple_setitem (ptuple, 2, py_buildvalue ("S", "Python"); // parse tuple itemsint I; float F; char * s; If (! Pyarg_parsetuple (ptuple, "ifs", & I, & F, & S) pyerr_setstring (pyexc_typeerror, "invalid parameter"); // cleanuppy_decref (ptuple );

 

2.3.3 list

The list in python is a variable-length array. The list is more flexible than the tuples. You can use the list to randomly access the python objects stored in it. The following example demonstrates how to use the python list type in the C language:

Example 4: typelist. C // create the listpyobject * plist = pylist_new (3); // new referenceassert (pylist_check (plist); // set some initial valuesfor (INT I = 0; I <3; ++ I) pylist_setitem (plist, I, py_buildvalue ("I", I); // insert an itempylist_insert (plist, 2, py_buildvalue ("S ", "inserted"); // append an itempylist_append (plist, py_buildvalue ("S", "appended"); // sort the listpylist_sort (plist ); // reverse the listpylist_reverse (plist); // fetch and manipulate a list slicepyobject * pslice = pylist_getslice (plist, 2, 4); // new referencefor (Int J = 0; j <pylist_size (pslice); ++ J) {pyobject * pvalue = pylist_getitem (plist, J); Assert (pvalue);} py_decref (pslice ); // cleanuppy_decref (plist );

 

2.3.4 dictionary

The dictionary in python is a data type accessed Based on keywords. The following example demonstrates how to use the python dictionary type in the C language:

Example 5: typedic. C // create the dictionarypyobject * pdict = pydict_new (); // new referenceassert (pydict_check (pdict); // Add a few named valuespydict_setitemstring (pdict, "first ", py_buildvalue ("I", 2003); pydict_setitemstring (pdict, "second", py_buildvalue ("F", 3.14f )); // enumerate all named valuespyobject * pkeys = pydict_keys (); // new referencefor (INT I = 0; I <pylist_size (pkeys); ++ I) {pyobject * pkey = pylist_getitem (pkeys, I); pyobject * pvalue = pydict_getitem (pdict, pkey); Assert (pvalue);} py_decref (pkeys ); // remove a named valuepydict_delitemstring (pdict, "second"); // cleanuppy_decref (pdict );

 

Back to Top

 

After learning about the C language interfaces of Python, you can use these interfaces provided by the python interpreter to compile C language extensions of Python. Assume that the following C language functions are supported:

Example 6: example. CINT fact (int n) {If (n <= 1) return 1; else return N * fact (n-1 );}

 

This function is used to calculate the factorial of a given natural number. If you want to call this function in the python interpreter, you should first implement it as a module in Python, you need to write the corresponding encapsulation interface as follows:

Example 7: Wrap. C # include <python. h> pyobject * wrap_fact (pyobject * Self, pyobject * ARGs) {int N, result; If (! Pyarg_parsetuple (ARGs, "I: Fact", & N) return NULL; Result = fact (n); Return py_buildvalue ("I", result );} static pymethoddef examplemethods [] = {"fact", wrap_fact, meth_varargs, "caculate n! "},{ Null, null }}; void initexample () {pyobject * m; M = py_initmodule (" example ", examplemethods );}

 

A typical Python extension module consists of at least three parts: the export function, method list, and initialization function.

To use a function in C language in the python interpreter, you must first compile the corresponding export function for it. In the preceding example, the export function is wrap_fact. In Python C language extensions, all exported functions have the same function prototype:

PyObject* method(PyObject* self, PyObject* args);

This function is an interface between the python interpreter and the C function. It has two parameters: Self and args. The self parameter is used only when the C function is implemented as an inline method (built-in method). Generally, the value of this parameter is null ). The args parameter contains all the parameters that the python interpreter must pass to the C function. The Python C Language extension interface provides the pyarg_parsetuple () function to obtain these parameter values.

All export functions return a pyobject pointer. If the corresponding C function does not return a real value (that is, the return value type is void), a global none object (py_none) should be returned ), add the reference count to 1, as shown below:

PyObject* method(PyObject *self, PyObject *args) { Py_INCREF(Py_None); return Py_None;}

 

The method list shows all the methods that can be used by the python interpreter. The method list corresponding to the above example is:

static PyMethodDef exampleMethods[] = { {"fact", wrap_fact, METH_VARARGS, "Caculate N!"}, {NULL, NULL}};

 

Each item in the method list consists of four parts: method name, export function, parameter transfer method, and method description. The method name is the name used to call this method from the python interpreter. The parameter passing method specifies the specific form of passing parameters to the C function in Python. The two optional methods are meth_varargs and meth_keywords. meth_varargs is the standard form of parameter passing, it uses the python tuples to pass parameters between the python interpreter and the C function. If meth_keyword is used, then, the python interpreter and the C function will pass parameters between the two through the python dictionary type.

All Python extension modules must have an initialization function so that the python interpreter can correctly initialize the module. The Python interpreter specifies that the names of all initialization functions must start with init and add the module name. For the module example, the corresponding initialization function is:

void initexample() { PyObject* m; m = Py_InitModule("example", exampleMethods);}

 

When the python interpreter needs to import the module, it will find the corresponding initialization function based on the module name. Once it is found, it will call the function for initialization, the initialization function registers all the methods available in this module with the python interpreter by calling the function py_initmodule () provided by the python C Language extension interface.

To use an extension module written in C language in the python interpreter, it must be compiled into a dynamic link library. The following uses RedHat Linux 8.0 as an example to describe how to compile the python extension module compiled in C into a dynamic link library:

[xiaowp@gary code]$ gcc -fpic -c -I/usr/include/python2.2 / -I /usr/lib/python2.2/config / example.c wrapper.c[xiaowp@gary code]$ gcc -shared -o example.so example.o wrapper.o

 

After the dynamic link library of the python extension module is generated, you can use the extension module in the python interpreter. This is the same as the python built-in module, the extension module is also used after being introduced through the import Command, as shown below:

[xiaowp@gary code]$ pythonPython 2.2.1 (#1, Aug 30 2002, 12:15:30)[GCC 3.2 20020822 (Red Hat Linux Rawhide 3.2-4)] on linux2Type "help", "copyright", "credits" or "license" for more information.>>> import example>>> example.fact(4)24>>>

 

Iv. Conclusion

As a powerful scripting language, python will be more widely used in various fields. To overcome the slow execution speed of the script language, Python provides the corresponding C Language extension interface, which is implemented by using the C language to implement the key code that affects the execution performance, it can greatly improve the speed of running scripts written in Python to meet actual needs.

References

1. You can get to know everything about Python from the python (http://www.python.org) website.
2. You can find the official Python C/API documentation (http://www.python.org/doc/current/api/api.html) on the python website ).
3. You can find the official documentation for writing Python extension modules (http://www.python.org/doc/current/api/api.html) on the python website ).