How to generate egg in Python's setuptools framework

This article mainly introduces how to generate egg under the setuptools framework of Python. This article is from the official IBM Developer Technical Documentation. if you need it, you can refer to this article to introduce the content of the setuptools framework, it is a secondary project of PEAK and provides more simple package management and release functions than distutils.
Start

The setuptools module is very easy to avoid ". For example, if we download a package built using setuptools instead of distutils, the installation should work as we expected: usually python setup. py install is enough. To implement this function, a package bound with setuptools will contain a small bootstrap module ez_setup.py in the archive file. The only thing to note here is that ez_setup.py tries to download and install the required setuptools in the background -- of course, this requires a machine connected to the network. If setuptools has already been installed on a local machine, this background step does not need to be executed. However, if it needs to be manually installed, a lot of transparency will be lost. However, most systems now have an Internet connection. it is not difficult to perform several special steps for machines without a network connection.

The real advantage of setuptools is not the ability to implement distutils-although it does enhance the functionality of distutils and simplify the content in the setup. py script. The biggest advantage of setuptools is its enhanced package management capabilities. It can use a more transparent method to find, download, and install dependency packages. It can also switch between multiple versions of a package, these versions are installed on the same system. you can also declare the requirements for a specific version of a package. you can also use only one simple command to update to the latest version of a package. The most impressive thing is that even if some package developers have never considered any setuptools compatibility issues, we can still use these packages.

Next let's discuss it in detail.

Guide

The tool ez_setup.py is a simple script that can guide the rest of setuptools. The easy_install script provided in the complete setuptools package has the same functions as ez_setup.py. However, the former assumes that setuptools has already been installed, so it will skip the installation process behind the scenes. Both versions can accept the same parameters and switches.

The first step in this process is to download a small script ez_setup.py:
Listing 1. download the boot script

% wget -q http://peak.telecommunity.com/dist/ez_setup.py

Then, you can run the script without any parameters to install the rest of setuptools (if it is not used as a separate step, when other packages are installed for the first time, ). The following content is displayed (of course, it depends on the version used ):
Listing 2. guiding setuptools

% python ez_setup.pyDownloading http://cheeseshop.python.org/packages/2.4/s/ setuptools/setuptools-0.6b1-py2.4.egg#md5=b79a8a403e4502fbb85ee3f1941735cbProcessing setuptools-0.6b1-py2.4.eggcreating /sw/lib/python2.4/site-packages/setuptools-0.6b1-py2.4.eggExtracting setuptools-0.6b1-py2.4.egg to /sw/lib/python2.4/site-packagesRemoving setuptools 0.6a11 from easy-install.pth fileAdding setuptools 0.6b1 to easy-install.pth fileInstalling easy_install script to /sw/binInstalling easy_install-2.4 script to /sw/binInstalled /sw/lib/python2.4/site-packages/setuptools-0.6b1-py2.4.eggProcessing dependencies for setuptools

Complete. This is what we need to do to ensure that setuptools is installed on the system.

Installation package

For many Python packages, to install these packages, you need to pass the package name as a parameter to ez_setup.py or easy_install. Now that setuptools has been loaded using the boot script, you can use easy_install, which is simpler internally (in fact, it is slightly different from the version we selected.

For example, suppose you want to install the SQLObject package. The process is very simple, as shown in listing 3. Note that the message says SQLObject depends on a package named FormEncode. Fortunately, this will be well solved:
Listing 3. installing a typical package

% easy_install SQLObjectSearching for SQLObjectReading http://www.python.org/pypi/SQLObject/Reading http://sqlobject.orgBest match: SQLObject 0.7.0Downloading http://cheeseshop.python.org/packages/2.4/s/ SQLObject/SQLObject-0.7.0-py2.4.egg#md5=71830b26083afc6ea7c53b99478e1b6aProcessing SQLObject-0.7.0-py2.4.eggcreating /sw/lib/python2.4/site-packages/SQLObject-0.7.0-py2.4.eggExtracting SQLObject-0.7.0-py2.4.egg to /sw/lib/python2.4/site-packagesAdding SQLObject 0.7.0 to easy-install.pth fileInstalling sqlobject-admin script to /sw/binInstalled /sw/lib/python2.4/site-packages/SQLObject-0.7.0-py2.4.eggProcessing dependencies for SQLObjectSearching for FormEncode>=0.2.2Reading http://www.python.org/pypi/FormEncode/Reading http://formencode.orgBest match: FormEncode 0.5.1Downloading http://cheeseshop.python.org/packages/2.4/F/ FormEncode/FormEncode-0.5.1-py2.4.egg#md5=f8a19cbe95d0ed1b9d1759b033b7760dProcessing FormEncode-0.5.1-py2.4.eggcreating /sw/lib/python2.4/site-packages/FormEncode-0.5.1-py2.4.eggExtracting FormEncode-0.5.1-py2.4.egg to /sw/lib/python2.4/site-packagesAdding FormEncode 0.5.1 to easy-install.pth fileInstalled /sw/lib/python2.4/site-packages/FormEncode-0.5.1-py2.4.egg

As you can see from these messages, easy_install needs to find information about this package on www.python.org/pypi/, and then find the place where it can be downloaded (here the egg package is on cheeseshop.python.org; later I will introduce more about egg ).

You can not only install the latest version of a package (this is the default operation ). If you like, you can also provide a specific version requirement for easy_install. Now let's try to install a post-beta version of SQLObject.
Listing 4. installing the minimum version of a package

% easy_install 'SQLObject>=1.0'Searching for SQLObject>=1.0Reading http://www.python.org/pypi/SQLObject/Reading http://sqlobject.orgNo local packages or download links found for SQLObject>=1.0error: Could not find suitable distribution for Requirement.parse('SQLObject>=1.0')

If the latest version of SQLObject is less than 1.0, then nothing will be installed.

Install the "naive" package

SQLObject can recognize setuptools. But what if you want to install a package that is not compatible with setuptools? For example, before this article, I have never used setuptools for my "Gnosis Utilities. However, now let's try to install this package. it is known that only the HTTP (or FTP, SVN, CVS) location where it is located (setuptools can understand all these protocols ). My download Web site has various versions of Gnosis Utilities. their names adopt the common version style:
Listing 5. install a package that does not recognize setuptools

% easy_install -f http://gnosis.cx/download/Gnosis_Utils.More/ Gnosis_UtilsSearching for Gnosis-UtilsReading http://gnosis.cx/download/Gnosis_Utils.More/Best match: Gnosis-Utils 1.2.1Downloading http://gnosis.cx/download/Gnosis_Utils.More/ Gnosis_Utils-1.2.1.zipProcessing Gnosis_Utils-1.2.1.zipRunning Gnosis_Utils-1.2.1/setup.py -q bdist_egg --dist-dir /tmp/easy_install-CCrXEs/Gnosis_Utils-1.2.1/egg-dist-tmp-Sh4DW1zip_safe flag not set; analyzing archive contents...gnosis.__init__: module references __file__gnosis.magic.__init__: module references __file__gnosis.xml.objectify.doc.__init__: module references __file__gnosis.xml.pickle.doc.__init__: module references __file__gnosis.xml.pickle.test.test_zdump: module references __file__Adding Gnosis-Utils 1.2.1 to easy-install.pth fileInstalled /sw/lib/python2.4/site-packages/Gnosis_Utils-1.2.1-py2.4.eggProcessing dependencies for Gnosis-Utils

Fortunately, easy_install can complete all this well. It will view the given download directory, identify the highest available version, expand the package, and then repackage it into the "egg" format, which can be used for installation. Importing gnosis can now be run in a script. But what should I do if I want to test a script for a specific version earlier than Gnosis Utilities? This is also very simple:
Listing 6. installing a specific version of the "naive" package

% easy_install -f http://gnosis.cx/download/Gnosis_Utils.More/ "Gnosis_Utils==1.2.0"Searching for Gnosis-Utils==1.2.0Reading http://gnosis.cx/download/Gnosis_Utils.More/Best match: Gnosis-Utils 1.2.0Downloading http://gnosis.cx/download/Gnosis_Utils.More/ Gnosis_Utils-1.2.0.zip[...]Removing Gnosis-Utils 1.2.1 from easy-install.pth fileAdding Gnosis-Utils 1.2.0 to easy-install.pth fileInstalled /sw/lib/python2.4/site-packages/Gnosis_Utils-1.2.0-py2.4.eggProcessing dependencies for Gnosis-Utils==1.2.0

Currently, two versions of Gnosis Utilities are installed. The current active version is 1.2.0. Switching the active version back to 1.2.1 is also very simple:
Listing 7. modifying the "active" version within the system scope

% easy_install "Gnosis_Utils==1.2.1"Searching for Gnosis-Utils==1.2.1Best match: Gnosis-Utils 1.2.1Processing Gnosis_Utils-1.2.1-py2.4.eggRemoving Gnosis-Utils 1.2.0 from easy-install.pth fileAdding Gnosis-Utils 1.2.1 to easy-install.pth fileUsing /sw/lib/python2.4/site-packages/Gnosis_Utils-1.2.1-py2.4.eggProcessing dependencies for Gnosis-Utils==1.2.1

Of course, only one version can be active at a time. However, by adding these two lines of similar content to each script, you can let the script select the version you want to use:
Listing 8. using a certain version of the package in the script

from pkg_resources import requirerequire("Gnosis_Utils==1.2.0")

By using the above requirements, setuptools can add a specific version when running the import statement (if a value greater than the comparison is specified, it is the latest available version ).

Allow the package to recognize setuptools

I want users to install it without having to know the download directory of Gnosis Utilities. This usually works because Gnosis Utilities has an information List on Python Cheeseshop. Unfortunately, because setuptools is not taken into account, I created an "mismatched" portal for my Gnosis Utilities on python.org http://www.python.org/pypi/gnosis%20utilities/1.2.1. Specifically, this archive file is named based on a pattern similar to a Gnosis_Utils-N.N.N.tar.gz (these tools are also packaged. zip and .tar.bz2 files. The latest versions are also packaged into win32.exe installation programs, and all these files can be well processed by setuptools ). However, the spelling of the project name on Cheeseshop is slightly different from that on "Gnosis Utilities. In fact, a small administrative version Change in Cheeseshop will create the http://www.python.org/pypi/Gnosis_Utils/1.2.1-a as a post-release version. The archive file of the release has not changed, but a little metadata is added to the Cheeseshop. You only need a small amount of effort to use a simpler installer (note that for testing purposes, I ran easy_install-m to delete the installed package ).
Listing 9. simple addition of setuptools recognition

% easy_install Gnosis_UtilsSearching for Gnosis-UtilsReading http://www.python.org/pypi/Gnosis_Utils/Reading http://www.gnosis.cx/download/Gnosis_Utils.ANNOUNCEReading http://gnosis.cx/download/Gnosis_Utils.More/Best match: Gnosis-Utils 1.2.1Downloading [...]

I ignored the rest of the process, because it is no different from what you saw earlier. The only difference is that easy_install needs to find the metadata that matches the specified name on Cheeseshop (in other words www.python.org/pypi/) and use the information to find the real download location. In this case, the listed. ANNOUNCE file does not contain any helpful content, but easy_install will continue to view another listed URL, which will prove to be a download directory.

About egg

Egg is a package that contains all the package data. Ideally, egg is a zip compressed file, which includes all the required package files. However, in some cases, setuptools decides (or is notified by the switch) that the package should not be zip compressed. In these cases, egg is a simple uncompressed subdirectory, but the content is the same. A single version can be used for easy conversion and save a little disk space. However, the egg Directory has the same functions and organizational structure. Keep using the Java? Technical users will find egg very familiar.

In the latest Python version (2.3.5 + or 2.4 required), you can set PYTHONPATH or sys. path and import the corresponding package as usual to use egg. If you want to use this method, you do not need to use setuptools or ez_setup.py. For example, in the working directory used in this article, I put an egg for the PyYAML package. Now I can use this package. the method is as follows:
Listing 10. egg on PYTHONPATH

% export PYTHONPATH=~/work/dW/PyYAML-3.01-py2.4.egg% python -c 'import yaml; print yaml.dump({"foo":"bar",1:[2,3]})'1: [2, 3]foo: bar

However, PYTHONPATH (or sys. path in a script or Python shell session) is somewhat vulnerable. Egg discovery is best performed in the New. pth file. Any. pth file in site-packages/or PYTHONPATH will be parsed to perform other import operations. the method is similar to checking the directory locations that may contain packages. If you use setuptools to handle the management of a package, you need to modify a file named easy-install.pth when installing, updating, and deleting a package. In addition, you can name this. pth as long as its extension is. pth ). For example, below is the content of my easy-install.pth file:
Listing 11. the. pth file used for egg location configuration

% cat /sw/lib/python2.4/site-packages/easy-install.pthimport sys; sys.__plen = len(sys.path)setuptools-0.6b1-py2.4.eggSQLObject-0.7.0-py2.4.eggFormEncode-0.5.1-py2.4.eggGnosis_Utils-1.2.1-py2.4.eggimport sys; new=sys.path[sys.__plen:]; del sys.path[sys.__plen:]; p=getattr(sys,'__egginsert',0); sys.path[p:p]=new; sys.__egginsert = p+len(new)

This format is a bit special: It is similar to a Python script, but not completely. It should be noted that you can add the egg to be listed. in a better case, easy_install will implement this function at runtime. You can also create any number of. pth files under site-packages/. each file can list which egg is available.

Enhanced Installation Script

The ability to install the setuptools naive package (see listing 6) described above is only partially valid. That is to say, the package Gnosis_Utils is indeed installed, but not complete. All common functions work, but many Supported files are ignored when the egg is automatically generated-most of them are extensions. txt files and extensions are. xml test file (there are some other README ,. rnc ,. rng ,. xsl and files around sub-packages ). During installation, all these support files are "preferably available" without strict requirements. However, we still want to include all the support files.

The setup. py script used by Gnosis_Utils is actually very complicated. In addition to listing the basic metadata, in the 467th line of code, it also performs a complete test on the functions and bugs of the Python version; solves some faults in the earlier version of distutils; skip the installation of unsupported parts (for example, if pyexpat is not included in the Python release), process the OS line terminator conversion, and create multiple Archive/installer types; re-build the MANIFEST file based on the test results. Thanks to another maintainer of this package, Frank McIngvale, who can successfully install Gnosis_Utils back to Python 1.5.1, of course, the premise is that you need to do so (the features in earlier versions are not so rich ). However, the script I want to show you is not as complex as the distutils script: it simply assumes that a "normal" version of Python has been installed in the system. Even so, setuptools makes the installation script so simple and attractive.

In the first attempt, let's create a setup. py script, which is borrowed from the setuptools manual and used to create an egg:
Listing 12. setuptools setup. py script

% cat setup.pyfrom setuptools import setup, find_packagessetup(  name = "Gnosis_Utils",  version = "1.2.2",  packages = find_packages(),)% python setup.py -q bdist_eggzip_safe flag not set; analyzing archive contents...gnosis.__init__: module references __file__gnosis.doc.__init__: module references __file__gnosis.magic.__init__: module references __file__gnosis.xml.objectify.doc.__init__: module references __file__gnosis.xml.pickle.doc.__init__: module references __file__gnosis.xml.pickle.test.test_zdump: module references __file__

This effort can already take effect, at least partially. You can use these lines to create an egg. However, this egg is similar to the egg created using easy_install. it lacks support for files that do not use. py names. So let's try again, just a little more effort:
Listing 13. add the missing package_data

from setuptools import setup, find_packagessetup(  name = "Gnosis_Utils",  version = "1.2.2",  package_data = {'':['*.*']},  packages = find_packages(),)

This is all the operations that need to be done. Of course, you usually want to adjust it based on the actual situation. For example, it may list the following content:
Listing 14. package a specific type of file

package_data = {'doc':['*.txt'], 'xml':['*.xml', 'relax/*.rnc']}

This section is translated as follows. the txt file is included in the doc/sub-package. the xml file is included in the xml/sub-package. the rnc file is included in the xml/relax/sub-package.

Conclusion

This article only introduces the surface layer of custom operations that can be performed with the release that supports setuptools. For example, assume that you have a release (you can select the preferred egg format or another archive type ), you can use a command to upload the archive file and metadata to the Cheeseshop. Apparently, the complete setup. the py script should contain the same detailed metadata contained in the earlier distutils script. for the sake of simplicity, this article skips this content, but its parameter name is compatible with distutils.

Although it takes some time to fully adapt to the huge functions provided by setuptools, it does make it easier to maintain your own package and install external packages than distutils. If you only care about the installation package, all the content you need to know is included in this article; you may find some complexity when describing your own package, but it is still not as complicated as using distutils.