Introduction to the Python ecosystem and python Ecosystem

Introduction to the Python ecosystem [go] and introduction to the python Ecosystem
Introduction to the Python Ecosystem

Author:	Mir Nazim
Original article:	Python Ecosystem-An Introduction
Translator:	Dccrazyboy
2:	Introduction to the Python Ecosystem

When developers switch from PHP, Ruby, or other development environments to Python, the biggest problem is lack of full understanding of the Python development ecosystem. Developers really want a guide or resource on completing most tasks, regardless of whether the methods used are standard or not.

The content mentioned below basically comes from my website, which stores some basic information about network application development in the Python environment, these materials are prepared for interns, graduate students, and experienced developers who have switched from other platforms to Python development.

This is not a perfect document. My goal is to make it intoPermanent documentI hope this document can be developed into a detailed tutorial.

Target readers

This is not a grammar book about Python. This tutorial will not teach you how to use Python in a fancy way to make you a Python hacker. By default, you already know the basic usage of Python. If you don't know, don't read it down. Let's take a look at Zed Shaw's free book "Learn Python The Hard Way", which teaches Python beginners.

I don't know if you are using Linux (preferably Ubuntu/Debian) or a Linux-like system. Is it Mao? Because Linux is the most familiar system. Apart from cross-browser compatibility testing, I have never developed it on Windows or Max OS X. The following is a tutorial on how to install Python on different platforms.

 
 

  
  
Python 101: Setting up Python on Windows
  
  
Official documentation for Python on Windows
  
  
Official documentation for Python on Mac OS X
 
 

Search for the Python installation method that best suits your platform. I strongly asked Stack Overflow.

Version puzzles

Python 2.x is a stable version, and Python 3 is a new version. If you don't care, skip this section and check the Python installation section below.

When you start learning Python, installing a version of 3.x seems to be the first step, but it may not necessarily be what you want.

Currently, Python has two versions under development-2.7.x and 3.x( also known as Python3, Py3k or Python 3000 ). Compared with Python2, Python3 is another language. There are two or more major or small syntax differences between them. Today, Python2.6/2.7 is the most widely installed and applied version. Many mainstream codes and some important packages/frameworks/tools/utilities/modules do not support Python3.

Therefore, the safest choice is to use version 2.x( best version 2.7. If you fully understand Python3, use it again.

Python 3 Wall of Shame lists the compatibility of many packages in Python3. Take a good look at this before deciding to use Python3.

VM Selection

The Python interpreter or Python virtual machine has many implementations. CPython is the most popular implementation. CPython is also a reference interpreter for other virtual machine implementations.

PyPy is a Python interpreter implemented using Python, Jython is an interpreter running on JVM using Jave, and IronPython is an interpreter implemented using Microsoft. net clr.

Unless the interpreter is very important, we generally use CPython.

If the above nonsense about the version and virtual machine makes you a headache, you should install CPython 2.7.x. Believe me!

Install Python

In most Linux/Unix systems, Max OS X comes with Python. If no or the version is too low, run the following command to install it:

Ubuntu/Debian and its derivative systems:

$ sudo apt-get install python2.7

SudoIt is a Unix-like command that allows users to run programs with the permissions of other users (generally super users, or root users. For more information, see Wikipedia.

Fedora/Red Hat and similar systems:

sudo yum install python2.7

To use RHEL, you may need to open EPEL Repositories to install it.

From this point on, I will use sudo in the example, and you need to make changes based on your system.

Understanding package

First, you need to understand that Python does not have a default package management facility. In fact, the concept of packages is quite weak in Python.

As you may already know, Python code is organized as a module. A module may consist of a single file containing a function, or a directory containing multiple modules. The difference between a package and a module is very small, and each module can be understood as a package.

What is the difference between a package and a module (if any )? To understand this, you should first understand how Python looks for modules.

Like other programming environments, some functions and classes (such as str, len, and Exception) in Python are available globally (called built-in functions. You need to manuallyImportCome in. For example:

>>> import os>>> from os.path import basename, dirname

This package must exist on your machine so that it can be imported by the import Statement. But how does Python know the location of these modules? The location information is automatically set when you install the Python Virtual Machine and depends on your target platform.

You can query the package path in sys. path. The following is the result in my notebook. The running environment is Ubuntu 11.10.

>>> import sys>>> print sys.path['', '/usr/lib/python2.7', '/usr/lib/python2.7/plat-linux2', '/usr/lib/python2.7/lib-tk', '/usr/lib/python2.7/lib-old', '/usr/lib/python2.7/lib-dynload', '/usr/local/lib/python2.7/dist-packages', '/usr/lib/python2.7/dist-packages', '/usr/lib/python2.7/dist-packages/PIL', '/usr/lib/python2.7/dist-packages/gst-0.10', '/usr/lib/python2.7/dist-packages/gtk-2.0', '/usr/lib/pymodules/python2.7', '/usr/lib/python2.7/dist-packages/ubuntu-sso-client', '/usr/lib/python2.7/dist-packages/ubuntuone-client', '/usr/lib/python2.7/dist-packages/ubuntuone-control-panel', '/usr/lib/python2.7/dist-packages/ubuntuone-couch', '/usr/lib/python2.7/dist-packages/ubuntuone-installer', '/usr/lib/python2.7/dist-packages/ubuntuone-storage-protocol']

The path of the Python search package is provided here. It starts from the top until a name is found. This indicates that if two different paths contain two packages with the same name, the search will stop when the first name is found, and the search will never go down.

As you guessed, the package search path is easily hijacked. To ensure that Python loads your package first, you need to do the following:

>>> sys.path.insert(0, '/path/to/my/packages')

Although this method is useful in many cases, be careful not to abuse it.Use it only when necessary! Do not abuse it!

SiteThe search path of the module control package. When the Python virtual machine is initialized, it will be sub-imported. For more information, see the official documentation.

PYTHONPATH variable

PYTHONPATHIs an environment variable used to add the default package search directory. It can be considered as a special feature of Python.PATHVariable. It is just:Partition, including the list of Python module directories (not similarSys. pathPython list ). It may be like the following:

export PYTHONPATH=/path/to/some/directory:/path/to/another/directory:/path/to/yet/another/directory

Sometimes you may not want to overwrite existingPYTHONPATHTo add a new directory to the header or tail.

export PYTHONPATH=$PYTHONPATH:/path/to/some/directory    # Appendexport PYTHONPATH=/path/to/some/directory:$PYTHONPATH    # Prepend

PYTHONPATH,Sys. path. insertThese methods are not perfect, and we 'd better not use them. Using them, you may be able to solve local development environment problems, but it may not work in other environments. There are many ways to achieve this effect. I will explain it one by one below.

We now understand how to find the installation package path in Python. Now let's go back to the problem. What is the difference between modules and packages? A package is a collection of modules, modules, and submodules. It is usually compressed into a compressed package. This includes 1) Dependency Information 2) Instructions for copying files to the standard package search path. 3) Compile command (if the Code must be compiled before installation ). That's all!

Third-party packages

From the very beginning, if you want to do some practical Python development, you will certainly use some third-party packages.

There are at least three methods for installing third-party packages on Linux.

These three aspects almost accomplish the same thing. That is, install the dependency and compile the code (if needed). Search for a standard software package containing the module package copy.

Steps 2 and 3 are basically the same on all operating systems. I hope you can find the third-party package installation method on the Stack Overflow.

Where can I find a third-party package?

Before installing third-party packages, find them first. There are several methods:

Use a dedicated release package

Installing the installation package through the Package Manager is as simple as installing other software using the command line or GUI. For example, installSimplejson

$ sudo apt-get install python-simplejson

Install using pip

Easy_install is no longer popular. We will mainly introduce pip, which is a substitute for easy_install.

Pip is a tool used to install and manage Python packages, just like Python Packet Index. Pip is not installed with Python, so we need to install it first. In Linux, the installation is generally as follows:

$ sudo apt-get install python-pip

Before installing any other packages, I always upgrade pip to the latest version through PyPI. the versions in the Ubuntu software library are generally outdated. I upgraded pip myself through pip.

$ sudo pip install pip --upgrade

If you need to install any package, you can runPip install package-nameCommand. Therefore, you can run the following command to installSimplejson:

$ sudo pip install simplejson

Deleting a package is also easy.

$ sudo pip uninstall simplejson

Generally,PipThe latest stable version will be automatically installed from PyPI, but sometimes we need to install a specific version of the package, because your project may be based on a special version. Therefore, you may needPip installCommand:

$ sudo pip install simplejson==2.2.1

We may need to upgrade/downgrade/reinstall the installation package. In this case, you can run the following command:

$ sudo pip install simplejson --upgrade         # Upgrade a package to the latest version from PyPI$ sudo pip install simplejson==2.2.1 --upgrade  # Upgrade/downgrade a package to a given version

Now, if you want to install a package in the development version, which is available in the version control repository, but not in PyPI, what should you do?PipYou can handle this situation well, but before that, you need to install the version control repository on your own. In Ubuntu, you can install it as follows:

$ sudo apt-get install git-core mercurial subversion

After the version control warehouse is installed, the installation package from the version control warehouse is as follows:

$ sudo pip install git+http://hostname_or_ip/path/to/git-repo#egg=packagename$ sudo pip install hg+http://hostname_or_ip/path/to/hg-repo#egg=packagename$ sudo pip install svn+http://hostname_or_ip/path/to/svn-repo#egg=packagename

You can also install it from a local repository. Note that the following three slashes are file directories.

$ sudo pip install git+file:///path/to/local/repository

Note that if you useGitProtocol installation, you need to useGit + gitPrefix:

$ sudo pip install git+git://hostname_or_ip/path/to/git-repo#egg=packagename

Now you may be curious about this.EggHow to use it. Now you need to understand that an egg is a compressed Python package that contains source code and metadata.PipThe egg information is created before the installation package. You canSetup. pyFind the egg name in the file. FindSetupAnd then findName = "something". It may look like the following code (this code is from simplejson'sSrtup. py.

setup(name="simplejson", # <--- This is your egg nameversion=VERSION,description=DESCRIPTION,long_description=LONG_DESCRIPTION,classifiers=CLASSIFIERS,author="Bob Ippolito",author_email="bob@redivi.com",url="http://github.com/simplejson/simplejson",license="MIT License",packages=['simplejson', 'simplejson.tests'],platforms=['any'],**kw)

If noSetup. pyWhat about files? How can I find the egg name? We don't need it. Copy the package source code to your project directory, and import it to use it directly.

-- User option

All the above examples install the packages to the system scope. If you arePip installUse-- UserOption, the package will be installed~ /. LocalDirectory. On my machine, as shown below:

$ pip install --user markdown2Downloading/unpacking markdown2  Downloading markdown2-1.0.1.19.zip (130Kb): 130Kb downloaded  Running setup.py egg_info for package markdown2Installing collected packages: markdown2  Running setup.py install for markdown2    warning: build_py: byte-compiling is disabled, skipping.    changing mode of build/scripts-2.7/markdown2 from 664 to 775    warning: install_lib: byte-compiling is disabled, skipping.    changing mode of /home/mir/.local/bin/markdown2 to 775Successfully installed markdown2Cleaning up...

Note:Markdown2Installed/Home/mir/. local/bin/markdown2Under this directory.

There are many reasons why you don't want to install the package in the system directory. Later, I will explain how to set up an independent Python environment for each project.

Install from source code

Only one command is required from the source code installation package. decompress the package to a directory and then execute the following command.

cd /path/to/package/directorypython setup.py install

Although these installation methods are no differentPipThe best way.PipThis gives you the ability to easily upgrade/downgrade packages, and you have to manually download and decompress the package for manual installation. The manual installation package should make you the last option if all the other options fail (generally not possible ).

Install the package to be compiled

We have learned how to install most packages, but some packages are not introduced yet: Python packages that contain C/C ++ code must be compiled before installation. The best examples of these packages are database adapters and image processing libraries.

AlthoughPipYou can compile and install the source code, but I personally prefer to use the package provided by the Package Manager of the release version. It will install the compiled binary file.

If you still want to usePipInstall the SDK on Ubuntu.

Compiler tools:

$ sudo apt-get install build-essential

Python Development Environment (header files, etc ):

$ sudo aptitude install python-dev-all

If your system does notPython-dev-allLook at these similar namesPython-dev,Python2.X-devAnd so on.

Make sure you have installedPsycopg2(PostgreSQL RDBMS adapter for Python), you need PostgreSQL development files.

$ sudo aptitude install  postgresql-server-dev-all

After installing these dependencies, you can runPip install.

$ sudo pip install psycopg2

Note that:Not all packages can be compiled and installed through pip!. However, if you are confident in compilation and installation, or have sufficient experience in installing on your target platform. Then install it manually!

Python Development Environment

Different people prefer to build their own development environments in different ways, but in almost all programming communities, there is always one (or more) development environment that is more acceptable. Although there are no errors in using different development environments, some environment settings are easier to perform convenient tests and some repetitive/templated tasks are performed, makes daily work simple and easy to maintain.

Virtualenv

The most common method in the Python development environment is to use the virtualenv package. Virtualenv is a package used to create an independent Python environment. Now, the question is: why do we need an independent Python environment? To answer this question, allow me to reference virtualenv's own document:

The basic problems we need to deal with are package dependencies, versions, and indirect permissions. Imagine that you have two applications. One application requires version 1 of libfoo, And the other application requires version 2. How can I use these applications at the same time? If you install everything in/usr/lib/python2.7/site-packages (or any standard location on the platform), in this case, you may accidentally upgrade applications that should not be upgraded.

To put it simply, you can create different/independent Python environments for each project. You will install all required software packages for each project into their own environments.

UsePipCommand to install virtualenv:

$ sudo pip install virtualenv

After virtualenv is installed, run the following command to create an independent python environment for your project.

$ mkdir my_project_venv$ virtualenv --distribute my_project_venv# The output will something like:New python executable in my_project_venv/bin/pythonInstalling distribute.............................................done.Installing pip.....................done.

What happened above? You have created a folder.My_project_venvTo store your new independent Python environment.-- DistributeOption to make virtualenv use the new release-based package management system insteadSetuptoolsThe obtained package. What you need to know now is-- DistributeOption will be automatically installed in the new Virtual EnvironmentPipIn this way, you do not need to install it manually. When you become a more experienced Python developer, you will understand the details.

Now lookMy_project_venvDirectory, you will see this structure:

# Only the directories and files to be discussed are listed here. | -- bin | -- activate # <-- activation file of virtualenv | -- pip # <-- independent pip of virtualenv | '-- python # <-- a copy of the python Interpreter '-- lib' -- python2.7 # <-- all new packages will be

Run the following command to activate virtualenv:

$ cd my_project_venv$ source bin/activate

After the execution is completed, the prompt may look like this:

(my_project_venv)$ # the virtualenv name prepended to the prompt

Run the deactivate command to exit the virtualenv environment.

(my_project_venv)$ deactivate

Run the following command to better understand the differences between the two. If you have already entered virtualenv, please leave first.

First, let's take a look at the one that will be called if you call the python/pip command.

$ which python/usr/bin/python$ which pip/usr/local/bin/pip

Try again! Open virtualenv this time and see what is different. The following is displayed on my HOST:

$ cd my_project_venv$ source bin/activate(my_project_venv)$ which python/home/mir/my_project_venv/bin/python(my_project_venv)$ which pip/home/mir/my_project_venv/bin/pip

Virtualenv copies the Python executable file, creates some useful scripts, and installs the software packages required by the project, you can install, upgrade, or delete these packages throughout the project lifecycle. It also modifies some search paths, such as PYTHONPATH, to ensure that:

Also, by default, all packages installed within the system are visible to virtualenv. This means that if you install simplejson In Your Python directory, it will be automatically provided to all virtualenvs. This behavior can be changed and added when virtualenv is created.-- No-site-packagesThe virtualenv option does not read the system package, as shown below:

$ virtualenv my_project_venv --no-site-packages

Virtualenvwrapper

VirtualenvwrapperIsVirtualenvYou can easily create, activate, manage, and destroy a virtual environment using the tool. Without the tool, the above operations will be quite troublesome. You can run the following command to installVirtualenvwrapper.

$ sudo pip install virtualenvwrapper

After installation, You need to configure it. The following is my Configuration:

if [ `id -u` != '0' ]; then  export VIRTUALENV_USE_DISTRIBUTE=1        # <-- Always use pip/distribute  export WORKON_HOME=$HOME/.virtualenvs       # <-- Where all virtualenvs will be stored  source /usr/local/bin/virtualenvwrapper.sh  export PIP_VIRTUALENV_BASE=$WORKON_HOME  export PIP_RESPECT_VIRTUALENV=truefi

SetWORKON_HOMEAndSource/usr/local/bin/virtualenvwrapper. shOnly a few lines of code are required, and other parts are added according to my personal preferences.

Add the above configuration~ /. BashrcAnd then run the following command once:

$ source ~/.bashrc

If you close all shell windows and labels, and then open a new shell window or tag,~ /. BashrcIt will also be executed and will automatically update yourVirtualenvwrapperConfiguration. The result is the same as running the preceding command.

To create, activate, disable, and delete a virtual space, run the following command:

$ mkvirtualenv my_project_venv$ workon my_project_venv$ deactivate$ rmvirtualenv my_project_venv

Tab completion is available in virtualenvwrapper ~

Go to the virtualenvwrapper homepage to find