Python Software directory structure specification

Why design a directory structure?

The design project directory structure, like code coding style, belongs to the personal style problem. There are always two approaches to this style of regulation:

1. One class of students believes that this personal style problem is "irrelevant". The reason is that the program work is good, the style problem is not a problem at all.
2. Another class of students think that standardization can better control procedures structure, so that the program has a higher readability.

I am more inclined to the latter, because I am a former class of students under the direct victim of thought behavior. I have maintained a very difficult to read project, the logic of its implementation is not complicated, but it took me a very long time to understand what it meant to say. From then on, I personally have high requirements for improving the readability and maintainability of the project. "Project directory Structure" is also part of the "readability and maintainability" category, we design a clear hierarchy of directory structure, is to achieve the following two points:

1. High readability: People who are unfamiliar with the code for this project can read the directory structure at a glance, know which program startup script is, where the test directory is, where the configuration file is, and so on. To get to know this project very quickly.
2. High maintainability: Once you have defined your organization's rules, the maintainer will be able to clearly know which file and code should be placed under what directory. The benefit is that as time goes on, the size of the code/configuration increases and the project structure is not cluttered and can still be well organized.

So, I think it is necessary to maintain a clear hierarchy of directories. Not to mention the organization of a good project directory, in fact, is a very simple thing.

How directory is organized

There are already some common directory structures for how to organize a better Python project directory structure. On this issue of StackOverflow, we can see the discussion of the Python directory structure.

This is already very good, I do not intend to rebuild the wheel to enumerate the different ways, which I say my understanding and experience.

Assuming that your project is named Foo, I would recommend the most convenient and quick directory structure that would suffice:

Foo/|-- bin/|   |-- foo||-- foo/|   |-- tests/|   |   |-- __init__.py|   |   |-- test_main.py|   ||   |-- __init__.py|   |-- main.py||-- docs/|   |-- conf.py|   |-- abc.rst||-- setup.py|-- requirements.txt|-- README

Briefly explain:

1. bin/: Store some executable files of the project, of course you can name and script/ so on.
2. foo/: stores all source code for the project. (1) All modules and packages in the source code should be placed in this directory. Do not place the top level directory. (2) Its subdirectory tests/ holds unit test code, and (3) The entrance of the program is preferably named main.py .
3. docs/: Store some documents.
4. setup.py: Install, deploy, and package the scripts.
5. requirements.txt: A list of external Python packages that store software dependencies.
6. README: Project description file.

In addition, there are a number of scenarios that give more content. For example LICENSE.txt , ChangeLog.txt files and so on, I am not listed here, because these things are mainly open source projects need to use. If you want to write an open source software, how to organize the directory, you can refer to this article.

Now, let me briefly explain my understanding of these directories and my personal requirements.

About the content of the Readme

This is a document that I think should be available for each project, with the aim of briefly describing the project's information and giving the reader a quick overview of the project.

It needs to illustrate several things:

1. Software positioning, the basic functions of the software.
2. How to run your code: Installation environment, startup commands, and so on.
3. Brief instructions for use.
4. Code directory structure Description, more detailed can explain the basic principles of the software.
5. FAQ's description.

I think the above points are the better one README . In the early stages of software development, it is not always possible to complete all the information in the first place because the above content may be ambiguous or changed during the development process. But at the end of the project, it is necessary to write such a document.

can refer to the Redis source code in the wording of the readme, which is concise but clearly describes the Redis function and source structure.

About Requirements.txt and setup.pysetup.py

In general, use it setup.py to manage the packaging, installation, and deployment of your code. The industry standard notation is to use Python's popular packaging tools Setuptools to manage these things. This approach is commonly used in open source projects. But the core idea here is not to use standardized tools to solve these problems, but to say that a project must have an installation deployment tool that can quickly and easily install the environment on a new machine, deploy the code, and run the program.

I've stepped on the pit.

When I first started working on Python writing projects, installing the environment, deploying the code, and running the program were all done manually and encountered the following issues:

1. Install the environment often forget that recently added a new Python package, the result of running on the line, the program is wrong.
2. Python package version dependencies, sometimes we use a version of the Python package, but the official is already the latest package, by manual installation may be wrong.
3. If you rely on a lot of packages, it is time-consuming to install these dependencies one by one.
4. When a new classmate starts writing a project, it is very troublesome to run the program because it may often forget how to install various dependencies.

setup.pyThese things can be automated to improve efficiency and reduce the probability of errors. "Complex things are automated and things that can be automated must be automated. "is a very good habit.

Setuptools documents are huge, just contact, it may not be very good to find an entry point. The way to learn technology is to see how others are used, you can refer to a Python web framework, How flask is written: setup.py

Of course, simply writing a setup script ( deploy.sh ) setup.py can be an alternative.

Requirements.txt

The purpose of this file exists is to:

1. Easy for developers to maintain software package dependencies. Add the new packages in the development process to this list to avoid setup.py missing packages when installing dependencies.
2. Make it easy for readers to identify which Python packages the project uses.

The format of this file is that each line contains a description of the package dependencies, usually flask>=0.10 this format, the requirement is that the format can be pip recognized, so that you can simply pass pip install -r requirements.txt all the Python package dependencies are installed. Specific format description: click here.

about how to use the configuration file note that in the above directory structure, there is no conf.pyPlaced in the source directory, but placed in docs/Directory.

Many projects use the configuration file as follows:

1. The configuration file is written in one or more Python files, such as the conf.py here.
2. Which module in the project uses this configuration file directly import conf in this form to use the configuration in the code.

I don't quite agree with this approach:

1. This makes unit testing difficult (because the module internally relies on external configuration)
2. On the other hand, the configuration file as the interface of the user control program, should be free to specify the path of the file.
3. The reusability of the program components is too poor, because the code is hardcoded across all modules, making most modules dependent on conf.py the file.

So, I think the use of configuration, the better way is that

1. The configuration of the module is flexible and is not affected by external configuration files.
2. The configuration of the program can also be flexibly controlled.

Can support this idea is, with Nginx and MySQL students know that nginx, MySQL these programs are free to specify the user configuration.

Therefore, you should not use the configuration file directly in your code import conf . In the above directory structure conf.py , is a configuration sample given, not to write dead in the program directly referenced by the configuration file. You can main.py let the program read the configuration by specifying the configuration path for the startup parameter. Of course, conf.py you can change a similar name here, for example settings.py . Or you can use other formats to write configuration files, such as settings.yaml .

Python Software directory structure specification