- Objective
- Reasons to extend the Python language:
- Steps to create a python extension
- 1. Create Application code
- 2. Use the template to package the code
- A. Header files that contain Python
- B. Add a wrapper function for each function of each module, such as pyobject* Module_func ()
- C. Add an array of type Pymethoddef modulemethods[] for each module
- D. Add module initialization function void Initmethod ()
- 3. Compiling and Testing
- A. Creating setup.py
- B. Compile and connect your code by running setup.py
- C. Commissioning
- Simple performance Comparison
Preface (update: More convenient to use in http://www.swig.org/tutorial.html) Most of the Python extensions are written in C, but are also easily ported to C + +. In general, all code that can be integrated or imported into other Python scripts can be called extensions. Extensions can be written in pure Python, or in a compiled language such as C or C + +. Even if the same architecture between two computers is best not to share binary files with each other, it is best to compile Python and extensions on their respective computers. Because even a slight difference between the compiler or the CPU. Official document http://docs.python.org/extending/windows.html reasons to need to extend the Python language: 1. Add/extra (non-Python) features that provide parts that are not available in the Python core functionality, such as creating new data types or embedding Python in other existing applications, must be compiled. 2. Performance bottleneck efficiency, interpretation of the language is generally slower than the compiled language, want to improve performance, all changed into a compiled language is not cost-effective, good practice is to do performance testing, identify performance bottlenecks, and then the bottleneck in the expansion of the implementation, is a relatively simple and effective approach. 3. Keep proprietary source code private, scripting language a common flaw is that both are executed source code, confidentiality is not. Transferring part of the code from Python to the compiled language preserves proprietary source code privacy. It is not easy to reverse engineer this, which is important when it comes to special algorithms, encryption methods, and software security. Another way to keep your code secret is to publish only the precompiled. PYc file, which is a compromise method. Step 1 To create the Python extension. Create Application code #
include<stdio.h>
#
include<stdlib.h>
#
include<string.h>
#
DefineBUFSIZE 10
intFac
intN) {
if(N < 2)
return1;
returnn * FAC (N-1);
}
Char*reverse (
Char*s) {
Register
CharT
Char*p = s;
Char*q = (s + (strlen (s)-1));
while(P < Q) {
t = *p;
*p++ = *q;
*q--= t;
}
returnS
}
intMain () {
CharS[bufsize];
printf ("4! = =%d\n ", FAC (4));
printf ("8! = =%d\n ", FAC (8));
printf ("12! = =%d\n ", FAC (12));
strcpy (S, "abcdef");
printf ("Reversing ' abcdef ', we get '%s ' \ n", reverse (s));
strcpy (S, "Madam");
printf ("Reversing ' madam ', we get '%s ' \ n", reverse (s));
return0;
It is generally necessary to write the main () function for unit testing using GCC to compile >GCC extest.c-o Extest execute >./extest 2. Using the template to wrap the code the entire extension is implemented around the concept of "wrapping". Your design should seamlessly combine your implementation language with Python as much as possible. The code of the interface is also known as the "boilerplate" code, which is an essential part of your code interacting with the Python interpreter: Our boilerplate code is divided into 4 steps: A. The header file containing Python needs to find out where Python's header file is, typically adding # include "Python.h" B to the above C code in/usr/local/include/python2.x. The use of a wrapper function that adds a type such as pyobject* Module_func () for each function of each module is to pass the value of Python to C first, and then convert the result of the function in C into a Python object and return it to Python. You need to add a static function for all functions that you want to be accessed by the Python environment, the return type is Pyobject *, and the function name is in the format of the module name _;
StaticPyobject * EXTEST_FAC (pyobject *self, Pyobject *args) {
intres;//Calculating the result value
intnum;//parameters
pyobject* retval;//return value
I indicates that the parameter type to be passed in is integer, and if so, the value is assigned to NUM, and if not, returns null;
res = pyarg_parsetuple (args, "I", &num);
if(!res) {
The wrapper function returns NULL, resulting in a typeerror exception in the python call
returnNULL;
}
RES = FAC (num);
The result of calculation in C needs to be transferred to a Python object, and I represents the integer object type.
retval = (Pyobject *) py_buildvalue ("I", res);
returnretval
} can also be written in a shorter, more readable form:
StaticPyobject * EXTEST_FAC (pyobject *self, Pyobject *args) {
intM
if(! (Pyarg_parsetuple (args, "I", &num))) {
returnNULL;
}
return(Pyobject *) Py_buildvalue ("I", FAC (num));
The following is a table of type conversion parameters for Python and C: There is also a py_buildvalue usage table: The reverse function is also packaged similarly:
StaticPyobject *
Extest_reverse (Pyobject *self, Pyobject *args) {
Char*orignal;
if(! (Pyarg_parsetuple (args, "s", &orignal))) {
returnNULL;
}
return(Pyobject *) Py_buildvalue ("s", Reverse (orignal));
You can also change the function that returns a tuple containing both the original string and the inverse string
StaticPyobject *
Extest_doppel (Pyobject *self, Pyobject *args) {
Char*orignal;
if(! (Pyarg_parsetuple (args, "s", &orignal))) {
returnNULL;
}
SS, you can return two strings, should reverse is on the original string operation, so you need to strdup copy first
return(Pyobject *) Py_buildvalue ("ss", orignal, Reverse (strdup (orignal)));
What's wrong with the code above? and C language-related issues, the more common is the memory leak ... In the above example, the Py_buildvalue () function generates a copy of the transferred data when it is generated to return the Python object. The above two strings are copied. But we applied for the memory to hold the second string and did not release it when we exited. So the memory is leaked. The right thing to do is to return the Python object, and then release the memory requested in the wrapper function.
StaticPyobject *
Extest_doppel (Pyobject *self, Pyobject *args) {
Char*orignal;
Char*reversed;
Pyobject * retval;
if(! (Pyarg_parsetuple (args, "s", &orignal))) {
returnNULL;
}
retval = (Pyobject *) Py_buildvalue ("ss", Orignal, Reversed=reverse (StrDup (orignal)));
Free (reversed);
returnretval
C. Add an array of type Pymethoddef modulemethods[] For each module we have created several wrapper functions that need to be listed somewhere, so that the Python interpreter can import and invoke them. This is what the modulemethods[] array needs to do. The format is as follows, each array contains information about a function, and the last array places two null values, representing the end of the declaration
StaticPymethoddef
Extestmethods[] = {
{"FAC", EXTEST_FAC, Meth_varargs},
{"Doppel", Extest_doppel, Meth_varargs},
{"Reverse", Extest_reverse, Meth_varargs},
{null, NULL},
}; The Meth_varargs represents the parameter in the form of a tuple. If we need to use the Pyarg_parsetupleandkeywords () function to parse the keyword parameters, this flag constant should be written as: Meth_varargs & Meth_keywords for logic and arithmetic. D. Add module initialization function void Initmethod () The last work is the initialization of the module. This part of the code is called when the module is imported by Python.
voidInitextest () {
Py_initmodule ("Extest", extestmethods);
The final code is as follows: #
include<stdio.h>
#
include<stdlib.h>
#
include<string.h>
#
include"Python.h"
#
DefineBUFSIZE 10
intFac
intN) {
if(N < 2)
return1;
returnn * FAC (N-1);
}
Char*reverse (
Char*s) {
Register
CharT
Char*p = s;
Char*q = (s + (strlen (s)-1));
while(P < Q) {
t = *p;
*p++ = *q;
*q--= t;
}
returnS
}
StaticPyobject *
EXTEST_FAC (Pyobject *self, Pyobject *args) {
intRes
intNum
pyobject* retval;
res = pyarg_parsetuple (args, "I", &num);
if(!res) {
returnNULL;
}
RES = FAC (num);
retval = (Pyobject *) py_buildvalue ("I", res);
returnretval
}
StaticPyobject *
Extest_reverse (Pyobject *self, Pyobject *args) {
Char*orignal;
if(! (Pyarg_parsetuple (args, "s", &orignal))) {
returnNULL;
}
return(Pyobject *) Py_buildvalue ("s", Reverse (orignal));
}
StaticPyobject *
Extest_doppel (Pyobject *self, Pyobject *args) {
Char*orignal;
Char*RESV;
Pyobject *retval;
if(! (Pyarg_parsetuple (args, "s", &orignal))) {
returnNULL;
}
retval = (Pyobject *) Py_buildvalue ("ss", Orignal, Resv=reverse (StrDup (orignal)));
Free (RESV);
returnretval
}
StaticPymethoddef
Extestmethods[] = {
{"FAC", EXTEST_FAC, Meth_varargs},
{"Doppel", Extest_doppel, Meth_varargs},
{"Reverse", Extest_reverse, Meth_varargs},
{null, NULL},
};
voidInitextest () {
Py_initmodule ("Extest", extestmethods);
}
intMain () {
CharS[bufsize];
printf ("4! = =%d\n ", FAC (4));
printf ("8! = =%d\n ", FAC (8));
printf ("12! = =%d\n ", FAC (12));
strcpy (S, "abcdef");
printf ("Reversing ' abcdef ', we get '%s ' \ n", reverse (s));
strcpy (S, "Madam");
printf ("Reversing ' madam ', we get '%s ' \ n", reverse (s));
Test ();
return0;
} 3. Compiling and testing in order for your new Python extensions to be created, you need to put them together with the Python library to compile. The Distutils package in Python is used to compile, install and distribute these modules, extensions and packages. The steps are as follows: A. Creating setup.py when we install the Python third-party package, we use the Python setup.py install command in many cases, let's look at the contents of the setup.py file. The main content of the compilation is done by the setup function, you need to create a extension instance for each extension, where we only have one extension, so we just need to create an instance. Extension (' Extest ', sources=[' extest.c ']), the first parameter is the name of the extension, and if the module is part of the package, add "." The second parameter is the source code file list Setup (' Extest ', ext_modules=[...]), the first parameter indicates which thing to compile, and the second parameter lists the extension object to compile. #!/usr/bin/env python
fromDistutils.core
ImportSetup, Extension
MOD = ' Extest '
Setup (Name=mod, Ext_modules=[extension (MOD, sources=[' extest.c ')]) the Setup function also has many options to set. Details can be found on the website. Http://docs.python.org/distutils/setupscript.html B. Run setup.py to compile and connect your code running commands in the shell >python setup.py Build when you make an error such as: Unable to find the Python.h file then you do not have to install the Python-dev package, you need to go to the official website to download the source package reload yourself compiled and installed Python. Python.h files usually appear in the/usr/include/python2.x folder, I do not have here anyway ... Only recompile a python ... I am now on the Linux system Python version is 2.6.6, I download a same version of the source code, you can also download a higher version. HTTP://WWW.PYTHON.ORG/DOWNLOAD/RELEASES/2.6.6/Extract Source Package > Tar xzf python-2.6.6.tgz> cd python-2.6.6.tgz compile and install Python >/configure--prefix=/usr/local/python2.6> make> sudo make install create a new link to compile python > sudo ln-sf/usr/local /python2.6/bin/python2.6/usr/bin/python2.6 testing, it is possible to use this method to run different versions of Python on Linux. Python.h files are also found under the/usr/local/python2.6/include/python2.6 path. After you run the compilation successfully, your extension will be created in the bulid/lib.* directory. You will see a. so file, which is a dynamic library file under Linux: C. Debugging you can test directly with Python code: #!/usr/bin/python
fromcTYPES
Import*
Import
OS
#需要使用绝对路径
Extest = Cdll. LoadLibrary (
OS. GETCWD () + '/extest.so ')
PrintEXTEST.FAC (4) You can also execute commands in the current directory, install to your Python path > python setup.py install successfully, and import the test directly:
Finally, it is important to note that the original C file has a main function, because a system can only have a main function, so in order not to conflict, you can change the main function to test function, and then with the extest_test () wrapper function, Then add the extestmethods array so that the test function can be called.
StaticPyobject *
Extest_test (Pyobject *self, Pyobject *args) {
Test ();
#返回空的话, use the following sentence
return(Pyobject *) Py_buildvalue ("");
} Simple performance comparison test code
ImportExtest
ImportTime
Start = Time.time ()
A = Extest.reverse ("ABCD")
Timec = Time.time ()-Start
Print' C costs ', Timec, ' The result is ', a
Start = Time.time ()
b =
List("ABCD")
B.reverse ()
b = ". Join (b)
Timepython = Time.time ()-start
Print' Python costs ', Timepython, ' The result is ', B running results can be seen, Python is not absolutely slower than C, but also to see the situation.
Python extension Implementation method--python and C mixed programming
