Easy-to-use instructions for extending the Python program using the C language

First, Introduction

Python is a powerful high-level scripting language, its power not only in its own function, but also in its good scalability, because of this, Python has begun to be favored by more and more people, and has been repeatedly successfully used in various large-scale software system development process.

Unlike other common scripting languages, Python programmers can use the API provided by the Python language to extend the functionality of Python using C or C + +, which makes it possible to take advantage of Python's easy-to-use syntax and functionality, and to get a C or C + + Almost the same execution performance. Slow execution is a common feature of almost all scripting languages, and it is an important factor that has been criticized, and Python skillfully solves this problem with the organic combination of C language, which greatly expands the application scope of scripting language.

When developing a real software system in Python, it is often necessary to use C + + to extend Python. The most common scenario is that there is already a library written in C that requires some of the library's features in the Python language, which can be achieved with the extended functionality provided by Python. In addition, because Python is essentially a scripting language, some features Python implementations may be difficult to meet the actual software system for the performance of the requirements, you can also use the extension provided by Python, the key sections of the code are implemented in C or C + +, thus providing the execution performance of the program.

This article mainly introduces the C language extension interface provided by Python, and how to use these interfaces and the C + + language to extend the functionality of Python, supplemented by specific examples of how to implement Python's functional extensions.

Second, the C language interface of Python

Python is a scripting language implemented in C, which has excellent openness and extensibility, and provides a convenient and flexible application interface (API) that enables C + + programmers to extend the capabilities of the Python interpreter at various levels. Before you can extend the functionality of Python using C/D + +, you must first grasp the C-language interface provided by Python interpretation.
2.1 Python object (Pyobject)

Python is an object-oriented scripting language in which all objects are represented in the Python interpreter as pyobject,pyobject structures that contain all the member pointers of the Python object and maintain the type information and reference count of the Python object. In Python's extended programming, once the Python object is processed in C or C + +, it means maintaining a pyobject structure.

In Python's C-language extension interface, most functions have one or more arguments for the Pyobject pointer type, and most of the return values are pyobject pointers.
2.2 Reference count

To simplify memory management, Python implements automatic garbage collection through a reference counting mechanism, and each object in Python has a reference count that counts how many times the object has been referenced in different places. Whenever a Python object is referenced, the corresponding reference count increases by 1, and whenever a Python object is destroyed, the corresponding reference is reduced by 1, and the Python object is actually removed from memory only if the reference count is zero.

The following example illustrates how the Python interpreter can use reference counting to manage Pyhon objects:

Example 1:refcount.py

Class RefCount:  # etc.r1 = RefCount () # Reference count = 1R2 = R1     # Reference count is 2del (R1)     # Reference count is 1del (R2)     # Reference count is 0, delete object

Proper maintenance of reference counts is a key issue when working with Python objects in C/s + +, and it is easy to handle bad memory leaks. Python's C-language interface provides some macros to maintain reference counts, most commonly using py_incref () to increase the reference count of Python objects by 1, and py_decref () to reduce the reference count of Python objects by 1.
2.3 Data types

Python defines six types of data: integer, float, string, tuple, list, and dictionary, and first understand how to convert between C and Python data types when using the C language to extend Python functionality.

2.3.1 Integer, floating-point, and string

In Python's C language extension, it is relatively easy to use the three data types of integer, float, and string, just know how to build and maintain them. The following example shows how to use Python in the C language for these three types of data:

Example 2:TYPEIFS.C

Build an integerpyobject* pInt = Py_buildvalue ("I", 2003); Assert (Pyint_check (pInt)); int i = Pyint_aslong (pInt); Py_decref (PINT);//Build a floatpyobject* Pfloat = Py_buildvalue ("f", 3.14f); Assert (Pyfloat_check (pfloat)); float F = Pyfloat_asdouble (pfloat); Py_decref (pfloat);//Build a stringpyobject* pstring = Py_buildvalue ("s", "Python"); Assert (Pystring_check (pstring); int nlen = pystring_size (pstring); char* s = pystring_asstring (pstring); Py_decref (pstring);

2.3.2 tuples

A tuple in the Python language is a fixed-length array, and all non-keyword (non-keyword) parameters are passed as tuples when the Python interpreter invokes a method in the C language extension. The following example demonstrates how to use Python's tuple types in C:

Example 3:typetuple.c

Create the tuplepyobject* ptuple = pytuple_new (3), Assert (Pytuple_check (ptuple)), Assert (Pytuple_size (ptuple) = = 3); /Set the Itempytuple_setitem (ptuple, 0, Py_buildvalue ("I", 2003)); Pytuple_setitem (ptuple, 1, Py_buildvalue ("F", 3.14f)); Pytuple_setitem (Ptuple, 2, Py_buildvalue ("s", "Python"));//Parse tuple itemsint i;float F;char *s;if (! Pyarg_parsetuple (ptuple, "ifs", &i, &f, &s))  pyerr_setstring (pyexc_typeerror, "invalid parameter"); /Cleanuppy_decref (ptuple);

2.3.3 List

The list in the Python language is a variable-length array, and the list is more flexible than the tuple, and the list can be used to randomly access the Python objects it stores. The following example demonstrates how to use the Python list type in C:

Example 4:typelist.c

Create the listpyobject* pList = pylist_new (3); New Referenceassert (Pylist_check (pList));//Set some initial valuesfor (int i = 0; i < 3; ++i)  Pylist_setitem (pLi St, I, Py_buildvalue ("I", i));//Insert an Itempylist_insert (PList, 2, Py_buildvalue ("s", "inserted"));//Append an Itempy List_append (PList, Py_buildvalue ("s", "appended"));//Sort the listpylist_sort (pList);//Reverse the Listpylist_ Reverse (pList);//Fetch and manipulate a list slicepyobject* Pslice = Pylist_getslice (PList, 2, 4); New Referencefor (int j = 0; J < pylist_size (Pslice); ++j) {Pyobject *pvalue = Pylist_getitem (PList, j); Assert (PVal UE);} Py_decref (Pslice);//Cleanuppy_decref (pList);

2.3.4 Dictionary

A dictionary in the Python language is a data type that is accessed based on a keyword. The following example demonstrates how to use the Python dictionary type in C:

Example 5:TYPEDIC.C

Create the dictionarypyobject* pdict = Pydict_new (); New Referenceassert (Pydict_check (pdict));//Add a few named Valuespydict_setitemstring (Pdict, "first",            Py_ Buildvalue ("I", 2003)); Pydict_setitemstring (Pdict, "second",            Py_buildvalue ("F", 3.14f));//Enumerate all named valuespyobject* Pkeys = Pydict_keys (); New Referencefor (int i = 0; i < pylist_size (Pkeys); ++i) {Pyobject *pkey = Pylist_getitem (Pkeys, i); Pyobject *pvalue = Pydict_getitem (pdict, PKey); ASSERT (PValue);} Py_decref (Pkeys);//Remove a named Valuepydict_delitemstring (Pdict, "second");//Cleanuppy_decref (PDICT);

C language extension of Python
3.1 Module Package

After understanding the C language interface of Python, you can use these interfaces provided by the Python interpreter to write the C language extension of Python, assuming the following C-language function:

Example 6:example.c

int fact (int n) {if (n <= 1)   return 1; else   return n * fact (n-1);}

The function is to calculate the factorial of a given natural number, if you want to invoke the function in the Python interpreter, you should first implement it as a module in Python, which requires writing the appropriate encapsulation interface as follows:

Example 7:WRAP.C

#include 
 
  
   
  pyobject* wrap_fact (pyobject* self, pyobject* args) {int n, result;  if (! Pyarg_parsetuple (args, "i:fact", &n))  return NULL; result = fact (n); return Py_buildvalue ("I", result);}  Static Pymethoddef examplemethods[] = {{"Fact", Wrap_fact, Meth_varargs, "Caculate n!"}, {NULL, null}};void initexample () {pyobject* m; m = Py_initmodule ("Example", Examplemethods);}
 
  

A typical Python extension should contain at least three parts: Export functions, method lists, and initialization functions.
3.2 Exporting Functions

To use a function in the C language in the Python interpreter, first write the appropriate export function for it, and the export function in the example above is wrap_fact. In Python's C language extension, all the exported functions have the same function prototypes:

Pyobject* method (pyobject* self, pyobject* args);

The function is an interface between the Python interpreter and the C function, with two parameters: Self and args. The parameter self is only used when the C function is implemented as an inline method (built-in), usually with a value of NULL (NULL). The parameter args contains all the parameters that the Python interpreter will pass to the C function, usually using the function pyarg_parsetuple () provided by the Python C extension interface to obtain these parameter values.

All the exported functions return a pyobject pointer, and if the corresponding C function does not have a true return value (that is, the return value type is void), then a global None object (Py_none) should be returned, and its reference count will be increased by 1, as follows:

Pyobject* method (Pyobject *self, Pyobject *args) {py_incref (py_none); return py_none;}

3.3 Method List

The method list shows all the methods that can be used by the Python interpreter, and the corresponding method list is:

Static Pymethoddef examplemethods[] = {{"Fact", Wrap_fact, Meth_varargs, "Caculate n!"}, {null, NULL}};

Each item in the method list consists of four parts: The method name, the export function, the parameter passing method, and the method description. The method name is the name used when calling the method from the Python interpreter. The parameter passing method specifies the specific form of the python to pass the parameter to the C function, the optional two ways are Meth_varargs and meth_keywords, in which Meth_varargs is the standard form of the parameter passing, It passes the parameters between the Python interpreter and the C function through a python tuple, and if Meth_keyword is used, the Python interpreter and the C function will pass the argument between the two through the Python dictionary type.
3.4 Initialization function

All Python extensions must have an initialization function so that the Python interpreter can initialize the module correctly. The Python interpreter specifies that all function names for the initialization function must begin with INIT and add the module's name. For module example, the corresponding initialization function is:

void Initexample () {pyobject* m; m = Py_initmodule ("Example", Examplemethods);}

When the Python interpreter needs to import the module, it will find the appropriate initialization function based on the name of the module and, once found, call the function to do the appropriate initialization, and initialize the function by calling the function Py_initmodule () provided by Python's C extension interface. To register all the available methods in the module with the Python interpreter.
3.5 Compiling links

To use the extension module written in C in the Python interpreter, you must compile it into the form of a dynamic-link library. The following is an example of how to compile a Python extension module written in C into a dynamic-link library, Redhat Linux 8.0:

[xiaowp@gary code]$ gcc-fpic-c-i/usr/include/python2.2 \          -i/usr/lib/python2.2/config \          example.c wrapper.c[ Xiaowp@gary code]$ gcc-shared-o example.so example.o wrapper.o

3.6 Introducing the Python interpreter

Once the dynamic-link library of the Python extension is generated, the extension can be used in the Python interpreter, as is the case with Python's own module, which is also introduced through the Import command, as follows:

[Xiaowp@gary code]$ Pythonpython 2.2.1 (#1, 2002, 12:15:30) [GCC 3.2 20020822 (Red Hat Linux Rawhide 3.2-4)] on Linu X2type "Help", "copyright", "credits" or "license" for more information.>>> import example>>> example.f Act (4) 24>>>

Iv. concluding remarks

As a powerful scripting language, Python will be more widely used in various fields. To overcome the slow execution of scripting languages, Python provides the appropriate C language extension interface, which, by implementing the C language with the key code that affects execution performance, greatly improves the speed at which scripts written in Python can run at runtime to meet actual needs.

