How to pack and publish Python modules _python

Objective

Yesterday, I packed my vasp file processing library and uploaded it to PyPI, and now I can install vaspy directly through Pip and Easy_install (also welcome to use vasp to do computational chemistry with the children's shoes plus star and participate in),

Vaspy's Gothub Address: https://github.com/PytLab/VASPy
Vaspy's PyPI Address: https://pypi.python.org/pypi/vaspy/

Because of their memory is really not how to, afraid of time to forget, so here in the iron and iron with their own vaspy program as an example of Python packaging and upload for the next summary.

Vaspy Package file Structure

First write the entire file structure of the Vaspy package, followed by the following is an example to explain:

vaspy/
├──license
├──manifest
├──manifest.in
├──readme.rst
├──requirements.txt Scripts
│  ├──change_incar_parameters.py
│  ├──create_inputs.py
│  └── ...
├──setup.cfg
├──setup.py
├──tests
│  ├──incar_test.py
│  ├──__init__.py
│  ├──oszicar_test.py
│ ├──outcar_test.py │ ├──testdata │ │  ├──contcar │ │ ├──dos_sum │ │ ├──elfcar │ │  └──
... │  └──
... └──vaspy
  ├──__init__.py
  ├──iter.py
  ├──matstudio.py
  └── ...
 
4 directories, files

Tools for packaging and installing third party packages

Here we need to use tools such as Setuptools and Pip to package and release their own packages and installation, if you need to build a wheel also need to install the wheel module. If the Python version >=2.7.9 or >=3.4,setuptools and Pip are already installed, you may need to update to the latest version

pip install -U pip setuptools

You can use package management tools, such as

Yum install pip
sudo apt-get install pip

Install via get-pip.py script, if detected wheel and Setuptools are installed automatically

python get-pip.py


Specific tool installation and introduction will not say more, you can refer to requirements for installing packages

The role of different files in the package

setup.py

This file is the most important file for packaging the entire project, which provides two main features:

The setup () function, in which parameters specify how to configure your own project.
Command-line tools, including packaging, testing, publishing, and more. Can be viewed through the following command;

python setup.py --help-commands

Setup.cfg

This file contains some of the default parameters at build time, such as--universal parameters when building Bdist_wheel

[Bdist_wheel]
Universal=1

This way the--universal parameter is used by default each time you pack, and the effect is similar:

python setup.py bdist_wheel --universal

Readme.rst

This first I was written with Markdown, packaging released to PyPI after the discovery PyPI does not support markdown rendering, the page is really a mess, and then use Restrutruedtext grammar to rewrite it again. After all, the markup language grammar can basically seconds to start, really not to find a template Bishi on the line.
Restructuretext's grammar rules refer to official documents: Quick Restructuredtext

In fact, there is another way is to use Pandoc to convert markdown to RST format, a convenient way is to use the Pyandoc module in the release of the automatic conversion.
Specific methods can refer to: Use Markdown README ' s in Python modules

Manifest.in

This file, when packaged, tells Setuptools that additional packaging of those files is required, such as the test data file for unit tests in my vaspy, which I use to include in the file. Of course readme,license These can also be packaged in with it.
Below is my own manifest.in content:

Include Readme.rst
include Requirements.txt
include LICENSE
recursive-include scripts *
Recursive-include Tests *

Specific grammatical rules can refer to: the manifest.in template

vaspy/

This folder is the package where the Vaspy source code resides.

tests/

This folder is also a child package, contains the unit test script, in order to be able to use the Python setup.py test for unit test, specifically added __init__.pys make it a package.

Parameters for Setup ()

Here are only a few of the parameters I use, the specific use of other parameters can be referred to: https://docs.python.org/3/distutils/setupscript.html

Name

versions = "vaspy"

is the name of the entire project and will be packaged with this name and version number.

Version

From Vaspy import __version__
Version = __version__

Description

is a short description of the project, generally a word is good, will be shown on the lower end of the name of the PyPI.

Long_description

is a long description, equivalent to a simple project, if the string is in RST format, PYPI will automatically render HTML display. Here you can read the contents of the Readme.rst directly.

Url

A connection to a package, usually a link on a github or a readthedocs link.

Packages

A list of child packages to include, Setuptools provides find_packages () to help us look for packages under the root path, a function distutil is not.

Setup_requires

This parameter defines the other dependencies (most basic) required for Vaspy installation and smooth running, and installs these dependencies when using PIP installation.
The difference between this parameter and Requirements.txt can be referred to: Install_requires vs Requirements files

Classifier

This parameter provides a series of classifications, which are placed in a different directory on the PyPI to classify the items.
Specific categories name and rule reference: Https://pypi.python.org/pypi?%3Aaction=list_classifiers

Test_suite

This parameter can help us to use

python setup.py test

To run unit tests, no longer need to write a separate script, such as run_tests.py to run unit tests.
An official explanation of this parameter:

A string naming a unittest. TestCase subclass (or a package or module containing one or more of them, or (a) of such a subclass), or naming a fun Ction that can is called with no arguments and returns a unittest. TestSuite. If The named Suite is a module, and the module has a additional_tests () function, it is called and the results are To the tests is run. If The named Suite is a package, any submodules and subpackages are recursively to the added test suite.

That is, this parameter can accept multiple types of arguments:

Receive UnitTest. TestCase subclass, we can say that all unit tests are written in a test case, and then import comes in and then you give Test_suite
Receives the function object, which has no arguments and returns a unittest. TestSuite. So we can write a function individually, combine multiple test cases into a single suite and then return the function import to Test_suite.

Module and package name, I just use this way, before my own tests are separate multiple scripts, so that I add a __init__.py can be turned into a package, the package name to Test_suite,setuptools will magically put all the tests under this package run side, So I'll add the new script directly when I add the test script later, and nothing else needs to be changed.

Operation Effect:

zjshao@shao-pc:/mnt/d/dropbox/code/centos_code/vaspy$ python setup.py test
running test
running Egg_info
creating Vaspy.egg-info
writing Vaspy.egg-info/pkg-info
writing top-level to names Level.txt
writing dependency_links to vaspy.egg-info/dependency_links.txt
writing manifest file ' Vaspy.egg-info/sources.txt '
reading manifest file ' vaspy.egg-info/sources.txt '
reading manifest ' Manifest.in '
writing MANIFEST file ' vaspy.egg-info/sources.txt '
running Build_ext
( Tests.incar_test. Incartest) Make
sure we can compare two Incar objects ... ok
correctly (test_eq. incartest)
Test __eq__ () function ... ok
...
Several outputs are omitted here
 
----------------------------------------------------------------------
Ran tests in 3.574s
 
OK

Publish your own Python package

1. First go to PyPI registered account

2. Configuration ~/.pypirc as follows:

[Distutils]
Index-servers =
  PyPI
  pypitest
 
[PyPI]
username:shaozhengjiang
Password:mypassword
 
[ Pypitest]
Username:shaozhengjiang
Password:mypassword

3. Then register and upload your own package to the test server

PyPI provides a test server that we can test on this test server.

python setup.py register -r pypitest


And then

python setup.py sdist upload -r pypitest


If there are no problems, we should not get any mistakes.

4. Upload to PyPI

If the above test succeeds, we can follow the same steps to register and upload the package.

Python setup.py register-r pypi
python setup.py sdist upload-r

Ok, then we can see our own bag on PyPI (https://pypi.python.org/pypi/vaspy/).