Python-Python (x, y)

Note: I would like to thank the original author for an article originally transferred from blog bus! I also made some modifications to improve the introduction of Pythonx and y.

Python three things

This section describes some Python-related tools and common problems for beginners.

 

Update 1st 2011.1.15

Update 2nd 2012.3.4

Introduction

Here I assume that you have read a Python tutorial and are familiar with the structure and syntax of Python. You have tried most Python statements in the Python interaction environment under the command line, I think Python is a good language to continue. This article will introduce the selection of related tools in the practical use of Python, including IDE, debugging kit, and third-party library management tools. In addition, problems that are easy to encounter in some Chinese environments, such as unicode encoding and decoding are also described. This article describes Python development in Windows. The purpose of this article is to share some useful experiences and examples. If you find any omissions in this article, please contact me. Thank you.

Python Introduction

Python is a computer programming language that has become popular in recent years. According to the official Python website, main features of Python include cross-platform, free, simple and easy to maintain. In my personal understanding, Python is a language suitable for most people. Because various types of third-party libraries are available, such as simple desktop programs, dynamic website development, image processing, and table processing, even small applications such as the automatic posting machine, after simple learning, can be made by themselves without deep programming experience.

Some popular Python tutorials

 

Dive into Python is intended for students with certain programming basics. In addition, there is a Dive into Python 3 tutorial for Python3.

 

Learn Python The Hard Way. The book mainly uses various exercises to Learn from students who have no programming experience.

 

Invent Your Own Computer Game With Python allows you to create a Game tutorial as soon as you get started.

 

The Python Tutorial is an orthodox and complete Tutorial in The official document.

 

How to think like a scientist: learn with Python. It is strongly recommended that students who are new to Python and programming, MIT textbooks, and are also used in CS courses of Nanke University.

 

Hello, World, a good book for children to learn programming, using Python as a programming language

 

If you haven't started to get familiar with Python or are not familiar with it, find a tutorial that you think you can continue learning. In my personal experience, Python is the most widely used language that I have learned so far, in fact, you can start using Python without having to fully understand it.

Python version selection. Other releases are entangled in Python 2 and 3?

The choice of Python 2 and 3 series may be annoying. In fact, the difference is very simple: Python 3.x is better in all aspects, but the syntax is not compatible with Python 2.x. Python 2.x has stopped development. However, many third-party libraries still do not support Python 3. Many tools described later in this article, such as ipython, currently only support Python 2.7.

I recommend that you select Python 2.7.2 now, because most third-party libraries and tools currently have simple installation packages for Python 2.7 and do not need to do too much processing by yourself. For details about 2.6 or 2.7, we recommend Python 2.7, compared with 2.6, some built-in packages do not need to be installed.

　 Release

Currently, in addition to the official installation version, Windows also provides:

ActivePython. the difference between this and official versions is that it provides additional libraries and documents and automatically sets the PATH environment variable (as described later)

Python (x, y) is a version that I have always used and recommended to others. From the name, we can see that this release comes with a lot of frequently-used libraries for scientific computing. In addition, there are a lot of frequently-used libraries, such as PyQt for desktop software interface creation, as well as document processing, common libraries such as exe file generation. There are also a large number of tools, such as IDE, chart tabulation tools, and enhanced interactive shell. Many of the software mentioned below are included in this release. In other aspects, Python (x, y) also comes with offline documents of all manually compiled libraries. Each minor version upgrade provides a separate patch. In general, it is a carefully maintained release version. We recommend that you install this version.

Development Tools

Before you start development, you should download an offline document. Download an HTML file on this page and decompress it. index.html is the document homepage. The Quick Search on the left of the page can also be used offline. If you have any questions about function functions or want to query modules, you can view them here.

Select PyDev as the IDE

The choice of the Python integrated development environment seems to have been a hard choice for a long time. After trying many tools, I found that Eclipse-based PyDev is definitely the most complete IDE. In addition to breakpoint debugging, the automatic completion of PyDev code may be the most powerful in this type of IDE.

If you have installed Python (x, y), PyDev is already on your machine. If not, follow this article for installation.

You need to pay attention to the settings. First, open PyDev, open Window-> Preferences in the menu, and find PyDev-> Editor-> Code Completion on the left of the pop-up dialog box. Here you can set the relevant information for Automatic completion of the Code. It can reduce Autocompletion delay to prompt code earlier, and hook the Request completion on series as much as possible, so that PyDev can provide as many prompts as possible. Then find the Interpreter Python tab. Here you can set the so-called Forced Buildins, which can force some third-party libraries to complete code completion. In my experience, most third-party libraries can perform basic completion after such settings. In the specific step, select the corresponding tab, click New, and enter the name of the required module.

After the setting, the overall effect is definitely better in the similar IDE:

IPython replaces Python Shell

When learning Python, you should have been familiar with Python Shell. You can input Python statements and return results immediately. IPython is an enhanced version of Python Shell. If you have installed Python (x, y), IPython is already on your machine. If not, download the Windows Installer here for installation. After installing this, you also need to install pyreadline so that IPython can enable highlighting and auto-complementing.

Then you can use it by entering ipython when you need python in the command line. Start IPython to see if the first difference is that it is colored. Let's take a look at some of its basic and practical functions. The first is automatic completion. One is simple keyword completion, and the other is object method and attribute completion. As an example, we first introduce the sys module, and then enter sys. (Note that there is a point). Press the tab key, IPython will list the methods and attributes of all sys modules. Because it is in interactive mode, the Python statement is actually executed at this time, so the completion of common objects is also very good.

Next, in the example above, we enter sys ?, The docstring and related information of the sys module are displayed. This is also a convenient feature in many cases.

IPython also has many convenient functions. You can refer to the document to find them. Here is another amazing feature. If your program is executed by the command line, enter
Python foo. py (most Python programs), you can use IPython to perform breakpoint debugging anywhere in your program! Add the following statement anywhere in your program:

from IPython.Shell import IPShellEmbed  IPShellEmbed([])()  

Run your program as usual. You will find that when the program runs to the place where the statement is inserted, it will be transferred to the IPython environment. You can try running some commands to find that the IPython environment is at the position of the program at the moment. You can view various variables in the current state one by one, call various functions, and output values of interest to help debugging. Then you can exit IPython as usual, and the program will continue to run. Naturally, the statements you run under IPython will also affect the subsequent running of the program.

 

I can see this method here. Imagine that this is like suspending a high-speed program. You can check and modify the running program and then let it continue running. Here is an example. For example, if you compile a Web page bot, you have to check its content every time you retrieve a page, and then try to get the address of the next page. With this technique, you can interrupt the program after retrieving the page, experiment with various processing methods, write the correct processing method back to your code, and then proceed to the next step. This kind of workflow can only be done in a dynamic language like Python.

The perfect combination of ipython and Linux

Of course, Ipython has a seamless connection with the Linux system. To briefly introduce how to use Ipython in Ubuntu, you must first install Ipython:

Root @ 3414-mainsrv :~ # Sudo apt-get install ipython

Then you can use it in the terminal. There is a function called magic:

In [1]: magic

IPython's 'Magic 'functions
======================================

The magic function system provides a series of functions which allow you
Control the behavior of IPython itself, plus a lot of system-type
Features. All these functions are prefixed with a % character, but parameters
Are given without parentheses or quotes.

NOTE: If you have 'automagic 'enabled (via the command line option or with
% Automagic function), you don't need to type in the % explicitly. By default,
IPython ships with automagic on, so you should only rarely need the % escape.

Example: typing '% cd mydir' (without the quotes) changes you working directory
To 'mydir', if it exists.

You can define your own magic functions to extend the system. See the supplied
Ipythonrc and example-magic.py files for details (in your ipython
Configuration directory, typically $ HOME/. ipython /).

You can also define your own aliased names for magic functions. In your
Ipythonrc file, placing a line like:

Execute _ IPYTHON _. magic_pf = _ IPYTHON _. magic_profile

Will define % pf as a new name for % profile.

You can also call magics in code using the ipmagic () function, which IPython
Automatically adds to the builtin namespace. Type 'ipmagic? 'For details.

For a list of the available magic functions, use % lsmagic. For a description
Of any of them, type % magic_name ?, E.g. '% cd? '.

 

Let's take an example. For example, a magic function called bg is used for computing in the background. We need to calculate the 10th power of 2, you just need to enter the following information and hand it over to the bg function to throw the calculation to the background for computing without affecting your continued operation. You only need to remember that the job serial number is 1 and the result is required, you only need to output the result in jobs [1 ].

In [10]: bg pow (2, 10)
Starting job #1 in a separate thread.

In [11]: print jobs [1]. result
1024

In addition, using the rehash function, you can use ipython as a shell, and you can use any command, such:

In [15]: ps aux | grep DarwinStreamingSrvr
Root 17496 0.0 0.0 2040 508 pts/0 S + sh-c ps aux | grep DarwinStreamingSrvr
Root 17498 0.0 0.0 7368 852 pts/0 S + grep DarwinStreamingSrvr

Pip manages third-party Libraries

A major advantage of Python is that it has a large number of third-party libraries, including references from various aspects. However, installing a third-party library may be annoying for those who do not have the necessary skills. In fact, Python third-party library installation and management have a unique and correct practice. This practice requires that you do not have to do anything else, as long as you enter the name of the installation library.

Setuptools is also included in Python (x, y. If not, install setuptools first, which is a software for installing third-party libraries. Select the corresponding version of Windows Installer for download and installation, open a command line window, enter:

  easy_install pip  

If the system prompts that the program cannot be found, it indicates that you have not set the environment variable. If you install the official Python installation package, you will certainly have this problem, and it is likely that it will not be corrected for the time being. This is a stubborn programmer. Right-click my computer-properties-advanced system settings-environment variables-add C: \ python2 * \ Scripts to the PATH group. The result of this operation is to enter a command in any command line, and the system will search for the content in the directory we set. Then execute the above command. After installation, We will discard setuptools and switch to pip. To install any library, you only need to find its name (no version number is required) and use pip to install it. For example, to install django, enter the following command:

  pip install django  

In fact, easy_install is similar to pip in the past. It is used to search for information and download and install PyPI in the official third-party library. The advantage of pip is that it supports more advanced functions, such as virtual environments. If the installation fails, no damaged libraries are left. More importantly, pip can be uninstalled. Run the following command to uninstall a library that was previously installed by pip. Continue with the above example and uninstall django now:

  pip uninstall django  

This is a function missing from setuptools. It should be noted that most pure Python libraries can use this method in Windows, but compilation of C language modules is generally unlikely to succeed. In this case, find the corresponding Windows installation package on the corresponding kudd website.

Winpdb

If you use PyDev, you can use its built-in breakpoint debugging. Winpdb provides a familiar debugging environment for users who use other simple editors for Python development. Not surprisingly, Winpdb is also in Python (x, y. Therefore, if Python (x, y) is installed, You can constantly explore the excellent tools that come with it. The usage method is simple. Assume that the program is named foo. py, enter:

  winpdb foo.py  

A window will pop up later, that is, a debug GUI that everyone is familiar. Note that you need to click the row where you want to set the breakpoint, click F9 to set the breakpoint, and the background color of the line will change to red, as shown in.

Encoding Problems

As a Chinese user, it is estimated that encoding is the most common problem for beginners of Python. Problems occur when both English and Chinese are available, and the error information is hard to understand. If you want to solve the problem, you do not know where to start. Fortunately, the coding problem is well avoided through preventive measures. The following describes the problems that are easy to encounter when processing Chinese and Unicode in Python.

Unicode encoding Basics

Here is a very simple introduction to coding knowledge. This part may not be accurate. If you know more about Unicode, please contact me for help.

As you can imagine, Unicode is a large table with all the characters in the world, such as English letters and Chinese characters. In fact, each word in the Unicode standard has a unique corresponding number, for example, the 'character in' corresponds to the hexadecimal 0x4E2D, the letter 'A' corresponds to the hexadecimal 0x0061. This number is determined by the Unicode Consortium organization. If this encoding is used to indicate characters, it is theoretically acceptable. In this way, each digit number can correspond to one character.

In reality, not every article uses all the characters in the world. For example, an English article only has English letters with some symbols. If Unicode is used for storage, each character will waste too much space. So there are various types of encoding. Encoding we can understand here that part of the Unicode (for example, all Chinese characters, or all Japanese) characters are identified by another symbol in some way. Common Chinese encodings include UTF8 and GBK. Take the '中' character as an example. UTF8 splits the Unicode number 0x4E2D corresponding to the '中' character into a combination of three numbers. [0xE4, 0xB8, 0xAD]. Only these connections are displayed as a "medium" character. For comparison, GBK encoding encodes the Unicode code 0x4E2D corresponding to the word "in" into a combination of two numbers.
[0xD6, 0xD0]. In the GBK encoding environment, only these two numbers are displayed as '中.

In the above example, if we put the UTF-8 encoded [0xE4, 0xB8, 0xAD] In the GBK environment, what will happen? The numbers are encoded with the characters 'zhong' under GBK [0xD6, 0xD0]
Is not displayed as '中. The three characters will be followed by the characters before and after the three, according to the GBK encoding rules to find whether there is a corresponding character. The result may show an irrelevant character. Sometimes it is a symbol or it is not displayed at all. In this case, garbled characters are generated.

String and Unicode in Python 2.x

In Python 2.x, there are two types of String characters, namely String and Unicode. The interfaces provided by the two are very similar. Sometimes they can be automatically converted, which is easy to mislead people. In Python 3, these two types are replaced by Bytes and String respectively. This name is more indicative of the nature of the two: strings in Python 2.x store byte sequences without encoding information, that is, strings store encoded sequences, but he does not know which encoding he uses. On the contrary, Unicode stores the encoded string information, which stores the Unicode numbers of the corresponding characters. Here we use a program to describe how to create a simple script named encoding. py. The Code is as follows:

#!/usr/bin/python  

# -*- coding: utf-8 -*-    

Strs = "this is Chinese"

Unis = "this is also Chinese". decode ("utf8 ")

print strs[0:2]  

print unis[0:2].encode('gbk')    

print len(strs)  

print len(unis)  

The first two lines will be explained later, that is, to limit the runtime environment and the encoding format of the script file. This script can be downloaded here. If you want to write it yourself, make sure that the script encoding is utf8 instead of anything else. The running result in Windows is here, I think the problem can be explained:

C:\SHARED\Dev\scripts>encoding.py    

Here we need to note that our program is UTF-8 encoded, and the main significance is that all directly Written string characters in this program (the string characters enclosed) it is encoded in UTF8 format. However, the command line in Windows is in the GBK environment. Strs is a String. In fact, the String character Directly written in the program in Python 2.x is of the string type (String literal is not considered here ). First, we directly output strs [0: 2] And get a garbled character (this character just happens to be a word ). As mentioned above, strings store String sequences without encoding information. Here, the first two numbers in strs are extracted and displayed. Because the command line environment is GBK encoded, the corresponding word here happens to be a word, but it has nothing to do with the original word.

Unis is obtained by a String calling the decode () method, which is the most basic way to obtain Unicode in Python 2.x. Because String does not know what encoding format it uses, it is our responsibility to determine which encoding method it uses. We know that the encoding format in the Code is UTF8, so we can call the decode () method of String for anti-encoding, that is, decoding, converts a string from an encoded format to its unique Unicode number. Unis is the decoded result. In Python 2.x, the corresponding type is Unicode, and the Unicode number corresponding to each character is stored.

We try to output the first two characters of unis. Here we call the Unicode encode () method. This is the encoding process. We know that the encoding in Windows command line is GBK. Only characters encoded in GBK can be correctly displayed. So here we call the Unicode encode () method to encode the Unicode numbers stored in the unis according to the GBK rules and output them to the screen. Here we can see that the first two characters in unis are correctly displayed. Note that if you print Unicode directly in the command line, Python will automatically encode it based on the current environment and then display it again, but this masks the difference between the two. We recommend that you always manually call the encode and decode methods so that you will be aware of them.

The difference between the two is also evidence of our previous example.
In strs, the serial number sequence after UTF-8 encoding is stored. As shown above, a Chinese character after UTF-8 encoding becomes three consecutive characters, so the strs length is 3x4 = 12. You can imagine that strs stores a series of meaningless bit sequences instead of Chinese characters, While unis stores the corresponding Chinese Unicode encoding. We know that each character corresponds to a number, so the five words correspond to five numbers and the length is 5. Avoid and solve coding problems

After learning about the concepts of Python Unicode encoding and decoding, let's take a look at how to avoid annoying coding problems as much as possible.

First, if your code contains Chinese characters, you must declare the code encoding format. According to the introduction in PEP-0263, the encoding can be determined by adding the following two lines of comments at the beginning of the program:

#!/usr/bin/python  

# -*- coding: utf-8 -*-  

UTF-8 is the specified encoding format. In fact, you should always use UTF8 as the encoding format of Your Python program, because in the future, all Python 3 files will be UTF-8 encoded by default. In addition to the declaration, you must make sure that the editor you use to edit the Python program is really UTF8 encoded to store files.

Then we will develop a good habit of encoding and decoding. When your program has a String as the input, it should be converted to Unicode as soon as possible, and then processed in the program. When outputting the code, try your best. It is not until the final output time that Unicode is encoded as the String in the required encoding format. Similarly, you must ensure that all strings involved in the operation in your program are in Unicode format. Many well-known Python libraries, such as django, adopt this method, and the effect is quite good. Do not rely on Python to convert the two, nor put String and Unicode together. These operations are very prone to errors, and cannot be reproduced in Python 3. Although it is the programmer's responsibility to determine the String encoding format, sometimes you really do not know what encoding some String characters are. Here is a magic chardet that can help you. The following is an example taken from its page, which demonstrates its role: read any string of characters, guess its encoding format, and give a confidence.

>>> import urllib  

>>> urlread = lambda url: urllib.urlopen(url).read()  

>>> import chardet  

>>> chardet.detect(urlread("http://google.cn/"))  {'encoding': 'GB2312', 'confidence': 0.99}    

>>> chardet.detect(urlread("http://yahoo.co.jp/"))  {'encoding': 'EUC-JP', 'confidence': 0.99}   

>>> chardet.detect(urlread("http://amazon.co.jp/"))  {'encoding': 'SHIFT_JIS', 'confidence': 1}    

>>> chardet.detect(urlread("http://pravda.ru/"))  {'encoding': 'windows-1251', 'confidence': 0.9355}  

If confidence is very low, or chardet directly reports an error, most of which are strings that have been decoded multiple times by error code. You need to find a solution from other places.

If the introduction above does not allow you to understand the concept of Unicode, there are several articles about this issue:

This article introduces two Unicode articles: [1] and [2]. Unicode is explained in more detail.

Unicode In Python and Completely Demystified are described In detail for Unicode processing In Python.

Other resources except the preceding important issues.

Vim Python development resources

As a matter of fact, I am using Vim to write Python, and I feel pretty good. The following are related resources.

 

UltimateVimPythonSetup is a new Vim configuration file for Python.

 

You only need to search for Python and Vim in Vim as Python IDE.

 

All Vim color schemes of vimcolorschemetest are gathered here.

 

Python-related Vim plug-ins

Pythoncomplete. vim is configured according to the above description. Press Ctrl-X and Ctrl-O to automatically complete the input.
Python. vim enhances syntax highlighting.
Pyflakes. vim has a great syntax check to analyze your syntax to avoid low-level errors. Note that this is only useful in Vim7.2. If it is 7.1, there is no effect at all...

Other related resources

Use Python for scientific computing
I have basically explained all the modules in Python (x, y). I think foreigners certainly want to have an English version.

 

PyMOTW
This name looks like a Python Library (in fact it is really a...), but it is actually a document in general, "Python every Monday module ". The author introduces a library in the Python standard library every week for several years. You can regard it as a great supplement to the Python standard library documentation. When you look at the introduction in the standard library, you may wish to look for the corresponding introduction here. This is because the examples here are quite comprehensive, and basically the library you use is described here. Another good news is that PyMOTW has a great Chinese translation version.

 

Reddit.com/r/python and python.org planet
Python-related articles and resources. In my personal experience, I can see a lot of useful things every time.