"Go" extends Python's functionality in C language

Original Xiaowenpeng ([email protected]), original address: http://www.ibm.com/developerworks/cn/linux/l-pythc/

Pyton and C have their own advantages and disadvantages, the development of the program with Python fast, high reliability, and there are many ready-made modules to use, but the execution speed is relatively slow, C language is the opposite, it executes fast, but the development efficiency is low. In order to make full use of the advantages of the two languages, it is better to develop the whole software framework with Python, and implement its key modules in C language. This article describes how to use the C language to extend the functionality of Python, supplemented by specific examples of how to write an extension module for Python.

First, Introduction

Python is a powerful high-level scripting language, its power not only in its own function, but also in its good scalability, because of this, Python has begun to be favored by more and more people, and has been repeatedly successfully used in various large-scale software system development process.

Unlike other common scripting languages, Python programmers can use the API provided by the Python language to extend the functionality of Python using C or C + +, which makes it possible to take advantage of Python's easy-to-use syntax and functionality, and to get a C or C + + Almost the same execution performance. Slow execution is a common feature of almost all scripting languages, and it is an important factor that has been criticized, and Python skillfully solves this problem with the organic combination of C language, which greatly expands the application scope of scripting language.

When developing a real software system in Python, it is often necessary to use C + + to extend Python. The most common scenario is that there is already a library written in C that requires some of the library's features in the Python language, which can be achieved with the extended functionality provided by Python. In addition, because Python is essentially a scripting language, some features Python implementations may be difficult to meet the actual software system for the performance of the requirements, you can also use the extension provided by Python, the key sections of the code are implemented in C or C + +, thus providing the execution performance of the program.

This article mainly introduces the C language extension interface provided by Python, and how to use these interfaces and the C + + language to extend the functionality of Python, supplemented by specific examples of how to implement Python's functional extensions.

Back to top of page

Second, the C language interface of Python

Python is a scripting language implemented in C, which has excellent openness and extensibility, and provides a convenient and flexible application interface (API) that enables C + + programmers to extend the capabilities of the Python interpreter at various levels. Before you can extend the functionality of Python using C/D + +, you must first grasp the C-language interface provided by Python interpretation.

2.1 Python object (Pyobject)

Python is an object-oriented scripting language in which all objects are represented in the Python interpreter as pyobject,pyobject structures that contain all the member pointers of the Python object and maintain the type information and reference count of the Python object. In Python's extended programming, once the Python object is processed in C or C + +, it means maintaining a pyobject structure.

In Python's C-language extension interface, most functions have one or more arguments for the Pyobject pointer type, and most of the return values are pyobject pointers.

2.2 Reference count

To simplify memory management, Python implements automatic garbage collection through a reference counting mechanism, and each object in Python has a reference count that counts how many times the object has been referenced in different places. Whenever a Python object is referenced, the corresponding reference count increases by 1, and whenever a Python object is destroyed, the corresponding reference is reduced by 1, and the Python object is actually removed from memory only if the reference count is zero.

The following example illustrates how the Python interpreter can use reference counting to manage Pyhon objects:

Example 1: refcount.py
class refcount:
    # etc.
r1 = refcount () # The reference count is 1
r2 = r1 # The reference count is 2
del (r1) # reference count is 1
del (r2) # The reference count is 0, delete the object
When dealing with Python objects in C / C ++, proper maintenance of the reference count is a key issue, and poor processing will easily cause memory leaks. The C language interface of Python provides some macros to maintain the reference count. The most common is to use Py_INCREF () to increase the reference count of Python objects by 1 and Py_DECREF () to decrease the reference count of Python objects by 1.

2.3 Data type
Python defines six data types: integer, floating point, string, tuple, list, and dictionary. When using C to extend Python's functions, we must first understand how to convert between C and Python data types. .

2.3.1 Integer, floating point and string

It is relatively simple to use the three data types of integer, floating point and string in the C language extension of Python. You only need to know how to generate and maintain them. The following example shows how to use these three data types of Python in C language:

Example 2: typeifs.c
// build an integer
PyObject * pInt = Py_BuildValue ("i", 2003);
assert (PyInt_Check (pInt));
int i = PyInt_AsLong (pInt);
Py_DECREF (pInt);
// build a float
PyObject * pFloat = Py_BuildValue ("f", 3.14f);
assert (PyFloat_Check (pFloat));
float f = PyFloat_AsDouble (pFloat);
Py_DECREF (pFloat);
// build a string
PyObject * pString = Py_BuildValue ("s", "Python");
assert (PyString_Check (pString);
int nLen = PyString_Size (pString);
char * s = PyString_AsString (pString);
Py_DECREF (pString);
2.3.2 Tuple

A tuple in Python is an array of fixed length. When the Python interpreter calls a method in the C language extension, all non-keyword parameters are passed in tuples. The following example demonstrates how to use Python's tuple type in C language:

Example 3: typetuple.c
// create the tuple
PyObject * pTuple = PyTuple_New (3);
assert (PyTuple_Check (pTuple));
assert (PyTuple_Size (pTuple) == 3);
// set the item
PyTuple_SetItem (pTuple, 0, Py_BuildValue ("i", 2003));
PyTuple_SetItem (pTuple, 1, Py_BuildValue ("f", 3.14f));
PyTuple_SetItem (pTuple, 2, Py_BuildValue ("s", "Python"));
// parse tuple items
int i;
float f;
char * s;
if (! PyArg_ParseTuple (pTuple, "ifs", & i, & f, & s))
    PyErr_SetString (PyExc_TypeError, "invalid parameter");
// cleanup
Py_DECREF (pTuple);
2.3.3 List

A list in Python is a variable-length array. A list is more flexible than a tuple. Using a list, you can randomly access the Python objects it stores. The following example demonstrates how to use Python's list type in C language:

Example 4: typelist.c
// create the list
PyObject * pList = PyList_New (3); // new reference
assert (PyList_Check (pList));
// set some initial values
for (int i = 0; i <3; ++ i)
    PyList_SetItem (pList, i, Py_BuildValue ("i", i));
// insert an item
PyList_Insert (pList, 2, Py_BuildValue ("s", "inserted"));
// append an item
PyList_Append (pList, Py_BuildValue ("s", "appended"));
// sort the list
PyList_Sort (pList);
// reverse the list
PyList_Reverse (pList);
// fetch and manipulate a list slice
PyObject * pSlice = PyList_GetSlice (pList, 2, 4); // new reference
for (int j = 0; j <PyList_Size (pSlice); ++ j) {
  PyObject * pValue = PyList_GetItem (pList, j);
  assert (pValue);
}
Py_DECREF (pSlice);
// cleanup
Py_DECREF (pList);
2.3.4 Dictionary

A dictionary in Python is a data type accessed based on keywords. The following example demonstrates how to use Python dictionary type in C language:

Example 5: typedic.c
// create the dictionary
PyObject * pDict = PyDict_New (); // new reference
assert (PyDict_Check (pDict));
// add a few named values
PyDict_SetItemString (pDict, "first",
                     Py_BuildValue ("i", 2003));
PyDict_SetItemString (pDict, "second",
                     Py_BuildValue ("f", 3.14f));
// enumerate all named values
PyObject * pKeys = PyDict_Keys (); // new reference
for (int i = 0; i <PyList_Size (pKeys); ++ i) {
  PyObject * pKey = PyList_GetItem (pKeys, i);
  PyObject * pValue = PyDict_GetItem (pDict, pKey);
  assert (pValue);
}
Py_DECREF (pKeys);
// remove a named value
PyDict_DelItemString (pDict, "second");
// cleanup
Py_DECREF (pDict);
 
Back to top

Third, Python's C language extension 3.1 module package
After understanding the Python C language interface, you can use these interfaces provided by the Python interpreter to write Python C language extensions, assuming the following C language function:

Example 6: example.c
int fact (int n)
{
  if (n <= 1)
    return 1;
  else
    return n * fact (n-1);
}
The function of this function is to calculate the factorial of a given natural number. If you want to call this function in the Python interpreter, you should first implement it as a module in Python. This requires writing the corresponding encapsulation interface, as follows:

Example 7: wrap.c
#include <Python.h>
PyObject * wrap_fact (PyObject * self, PyObject * args)
{
  int n, result;
  
  if (! PyArg_ParseTuple (args, "i: fact", & n))
    return NULL;
  result = fact (n);
  return Py_BuildValue ("i", result);
}
static PyMethodDef exampleMethods [] =
{
  {"fact", wrap_fact, METH_VARARGS, "Caculate N!"},
  {NULL, NULL}
};
void initexample ()
{
  PyObject * m;
  m = Py_InitModule ("example", exampleMethods);
}
A typical Python extension module should contain at least three parts: export function, method list, and initialization function.

3.2 Export function
To use a function in C language in the Python interpreter, first write the corresponding export function for it. The export function in the above example is wrap_fact. In the C language extension of Python, all exported functions have the same function prototype:

PyObject * method (PyObject * self, PyObject * args);
This function is the interface between the Python interpreter and the C function, with two parameters: self and args. The parameter self is only used when the C function is implemented as a built-in method, usually the value of this parameter is NULL. The parameter args contains all the parameters that the Python interpreter wants to pass to the C function. Usually, the function PyArg_ParseTuple () provided by the Python C language extension interface is used to obtain these parameter values.

All exported functions return a PyObject pointer. If the corresponding C function does not have a real return value (that is, the return type is void), it should return a global None object (Py_None) and increase its reference count by 1, as follows As shown:

PyObject * method (PyObject * self, PyObject * args)
{
  Py_INCREF (Py_None);
  return Py_None;
}
3.3 Method List
The method list gives all the methods that can be used by the Python interpreter. The corresponding method list for the above example is:

static PyMethodDef exampleMethods [] =
{
  {"fact", wrap_fact, METH_VARARGS, "Caculate N!"},
  {NULL, NULL}
};
Each item in the method list consists of four parts: method name, derived function, parameter passing method and method description. The method name is the name used when calling the method from the Python interpreter. The parameter transfer method specifies the specific form of Python to transfer parameters to the C function. The two optional methods are METH_VARARGS and METH_KEYWORDS, of which METH_VARARGS is the standard form of parameter transfer. Passing parameters between, if METH_KEYWORD is used, then the Python interpreter and C function will pass parameters between the two through the Python dictionary type.

3.4 Initialization function
All Python extension modules must have an initialization function so that the Python interpreter can properly initialize the module. The Python interpreter stipulates that the function name of all initialization functions must start with init and add the name of the module. For the module example, the corresponding initialization function is:

void initexample ()
{
  PyObject * m;
  m = Py_InitModule ("example", exampleMethods);
}
When the Python interpreter needs to import the module, it will search for the corresponding initialization function according to the name of the module. Once it is found, the function will be called to perform the corresponding initialization work. (), To register all available methods in this module with the Python interpreter.

3.5 Compile link
To use the extension module written in C language in the Python interpreter, it must be compiled into the form of a dynamic link library. The following uses RedHat Linux 8.0 as an example to introduce how to compile a Python extension module written in C into a dynamic link library:

[[email protected] code] $ gcc -fpic -c -I / usr / include / python2.2 -I /usr/lib/python2.2/config example.c wrapper.c
[[email protected] code] $ gcc -shared -o example.so example.o wrapper.o
3.6 Introducing the Python interpreter
After the dynamic link library of the Python extension module is generated, the extension module can be used in the Python interpreter. Like the module that comes with Python, the extension module is also used after being imported through the import command, as follows:

[[email protected] code] $ python
Python 2.2.1 (# 1, Aug 30 2002, 12:15:30)
[GCC 3.2 20020822 (Red Hat Linux Rawhide 3.2-4)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import example
>>> example.fact (4)
twenty four
>>>
 
Back to top

4. Conclusion
As a powerful scripting language, Python will be more widely used in various fields. In order to overcome the problem of the slow execution speed of the scripting language, Python provides a corresponding C language extension interface. By implementing the key code that affects the execution performance in C language, the speed of the script written in Python can be greatly improved at runtime So as to meet actual needs.

References
You can learn all about Python from the Python (http://www.python.org) website.
The official Python C / API documentation (http://www.python.org/doc/current/api/api.html) can be found on the Python website.
The official documentation for writing Python extension modules can be found on the Python website (http://www.python.org/doc/current/api/api.html).
【Transfer】 Extend Python's functions with C language