Detailed explanation of how to package and release the Python module, and detailed explanation of python

Detailed explanation of how to package and release the Python module, and detailed explanation of python

Preface

Yesterday, I packed my VASP File Processing library and uploaded it to PyPI, now you can install VASPy directly through pip and easy_install (also welcome to add stars and participate in the use of VASP for Computational chemistry ),

VASPy GotHub address: https://github.com/PytLab/VASPy
VASPy PyPI address: https://pypi.python.org/pypi/vaspy/

Because my memory is really bad, I am afraid I will forget it after a long time, so here I will take my VASPy program as an example to summarize the packaging and uploading of python.

VASPy package file structure

First, write the entire file structure attached to the VASPy package. The subsequent content is described in this example:

VASPy/├── LICENSE├── MANIFEST├── MANIFEST.in├── README.rst├── requirements.txt├── scripts│  ├── change_incar_parameters.py│  ├── create_inputs.py│  └── ...├── setup.cfg├── setup.py├── tests│  ├── incar_test.py│  ├── __init__.py│  ├── oszicar_test.py│  ├── outcar_test.py│  ├── testdata│  │  ├── CONTCAR│  │  ├── DOS_SUM│  │  ├── ELFCAR│  │  └── ...│  └── ...└── vaspy  ├── __init__.py  ├── iter.py  ├── matstudio.py  └── ... 4 directories, 54 files

Tools for packaging and installing third-party packages

Here, we need to use tools such as setuptools and pip to package, release, and install our own packages. If we need to build a wheel, we also need to install the wheel module. If the python version is> = 2.7.9 or> = 3.4, setuptools and pip have been installed, you may need to update them to the latest version.

pip install -U pip setuptools

You can use the package management tool, such

yum install pipsudo apt-get install pip

Install with a get-pip.py script and automatically install if you detect that wheel and setuptools Are Not Installed

python get-pip.py


For more information about tool installation and introduction, see requirements for installing packages.

Functions of different files in the package

Setup. py

This file is the most important file for packaging the entire project. It provides two main functions:

Setup () function. The parameter of this function specifies how to configure your own project.
Command Line tools, including packaging, testing, and publishing. You can use the following command to View Details;

python setup.py --help-commands

Setup. cfg

This file contains some default parameters during construction, such as the -- universal parameter during bdist_wheel construction.

[bdist_wheel]universal=1

In this way, the -- universal parameter is used by default during each package. The effect is similar:

python setup.py bdist_wheel --universal

README. rst

I first wrote it using markdown. After packaging and publishing it to PyPI, I found that PyPI does not support markdown rendering. The page is really messy, so I wrote it again using the reStrutruedText syntax. After all, the syntax of the markup language can be used in seconds, but it is impossible to find a template that is better than that of Hulu.
For more information about the reStructureText syntax, see Quick reStructuredText.

In fact, another method is to use pandoc to convert markdown to rst format, and the easy-to-use method is to use the pyandoc module to automatically convert it when it is released.
For details, refer to: Use Markdown README's in Python modules.

MANIFEST. in

During packaging, this file tells setuptools that additional files need to be packaged. For example, I use this file to include the unit test data file in my VASPy. Of course, README and LICENSE can also be packaged together.
The content of my MANIFEST. in is as follows:

include README.rstinclude requirements.txtinclude LICENSErecursive-include scripts *recursive-include tests *

For specific syntax rules, see The MANIFEST. in template.

Vaspy/

This folder is the package where the source code of vaspy is located.

Tests/

This folder is also a sub-package that contains the unit test script, in order to be able to use python setup. py test is used for unit testing. Specifically, _ init _ is added __. pys makes it a package.

Setup () Parameters

Here only describes the several parameters I use, the specific use of other parameters can refer to: https://docs.python.org/3/distutils/setupscript.html

Name

versions = "vaspy"

Is the name of the entire project. This name and version number will be used after packaging.

Version

from vaspy import __version__version = __version__

Description

It is a brief description of the project. A general sentence is good and will be displayed at the bottom of the pypi name.

Long_description

It is a long description, which is equivalent to a concise description of the project. If the string is in the rst format, PyPI will be automatically rendered to HTML display. The content in README. rst can be directly read here.

Url

Package connection, usually the link on GitHub or readthedocs.

Packages

List of sub-packages to be included. setuptools provides find_packages () to help us find packages in the root path. This function is not available in distutil.

Setup_requires

This parameter defines other Dependencies (the most basic) required for VASPy installation and smooth running. These dependencies are installed when pip is used.
For more information about the differences between this plugin and requirements.txt, see install_requires vs Requirements files.

Classifier

This parameter provides a series of categories. On PyPI, it will be put into different directories for classification.
Specific categories names and Rules reference: https://pypi.python.org/pypi? % 3 Aaction = list_classifiers

Test_suite

This parameter can help us use

python setup.py test

To run the unit test, you no longer need to write another script, such as run_tests.py, to run the unit test.
Official explanation of this parameter:

A string naming a unittest. testCase subclass (or a package or module containing one or more of them, or a method of such a subclass), or naming a function that can be called with no arguments and returns a unittest. testSuite. if the named suite is a module, and the module has an additional_tests () function, it is called and the results are added to the tests to be run. if the named suite is a package, any submodules and subpackages are recursively added to the overall test suite.

That is to say, this parameter can accept multiple types of parameters:

Receives the unittest. TestCase subclass. We can write all unit tests into a test case, import them in, and then upload them to test_suite.
Receives the function object. This function object does not have any parameters and returns a unittest. testSuite. in this way, we can write a function separately, combine multiple test cases into a suite, return the result, and then import the function to test_suite.

The module and package name are used in this way. In the previous test, multiple scripts are separated. In this way, I add a _ init __. py can be converted into a package, and the package name is passed to test_suite. setuptools will magically run all the tests in this package, in this way, I will add a new script directly when I add a test script later, and no other changes are required.

Running effect:

SHAO-PC:/mnt/d/Dropbox/Code/CentOS_code/VASPy $ python setup. py testrunning egg_infocreating vaspy. egg-infowriting vaspy. egg-info/PKG-INFOwriting top-level names to vaspy. egg-info/top_level.txtwriting dependency_links to vaspy. egg-info/dependency_links.txtwriting manifest file 'vaspy. egg-info/SOURCES.txt 'reading manifest file 'vaspy. egg-info/SOURCES.txt 'reading manifest template 'MANIFEST. in 'writing manifest file' vaspy. egg-info/SOURCES.txt 'running build_exttest_compare (tests. incar_test.InCarTest) Make sure we can compare two InCar objects correctly .... oktest_eq (tests. incar_test.InCarTest) Test _ eq _ () function .... OK... some outputs are omitted here ---------------------------------------------------------------------- Ran 22 tests in 3.574 s OK

Release your own python package

1. First, register an account with PyPI.

2. Configuration ~ /. Pypirc is as follows:

[distutils]index-servers =  pypi  pypitest [pypi]username:ShaoZhengjiangpassword:mypassword [pypitest]username:ShaoZhengjiangpassword:mypassword

3. Register and upload your package to the test server.

Pypi provides a test server on which we can perform tests.

python setup.py register -r pypitest


Then

python setup.py sdist upload -r pypitest


If there is no problem, we should not get any errors.

4. Upload to PyPI

If the above test is successful, we can follow the same steps to register and upload the package.

python setup.py register -r pypipython setup.py sdist upload -r pypi

OK, then we can see our own package on PyPI (https://pypi.python.org/pypi/vaspy.