Python encoding and python Encoding
Python Encoding
ASCII, Unicode, UTF-8, and gbk
Before specific description of Python encoding, first clear ASCII, Unicode, UTF-8, gbk is what exactly? Here is a brief introduction. Please refer to Baidu for details.
ASCII:Is currently the most commonSingle-byte encoding system. ASCII (only 1 ~ 127) Only English letters, numbers, and symbols are supported. For example, the ASCII code of A is 65 (decimal ).
Unicode: To address the limitations of traditional character encoding schemes, a unified and unique binary encoding is set for each character in each language, to meet the requirements of cross-language and cross-platform text conversion and processing. Unicode is the encoding of all symbols. Unicode is only oneSymbol SetIt only specifies the binary code of the symbol, but does not specify how the binary code should be stored.
UTF-8:It is a variable-length character encoding method for Unicode. It is one of the Unicode implementation methods and is a variable-length encoding method. It can use 1 ~ The four bytes indicate a symbol, and the length of the byte varies according to different symbols. UTF-8 encoding encodes a Unicode character into 1-6 bytes according to different numbers, common English letters are encoded into 1 byte, Chinese characters are usually 3 bytes, only uncommon characters are encoded into 4-6 bytes. If the text you want to transfer contains a large number of English characters, using UTF-8 encoding can save space, refer to the following:
Gbk:GBK is compatible with GB2312 after expansion based on the National Standard GB2312. The GBK text encoding is expressed in double bytes, that is, both Chinese and English characters are expressed in Double Bytes. To distinguish Chinese characters, set the highest bit to 1. GBK contains all Chinese characters and is a country code.
Python3 supports Unicode
From Python 3.0, all strings support Unicode and are saved in UTF-8 format by default. Therefore, print Chinese can be normal without special processing.
However, Python3 does not have the sys. setdefaultencoding () function. Therefore, the extension package of the original Python2 needs to adapt to this situation.
Python2 processes Unicode
Python2 by default, the source code file is ASCII encoded. Therefore, if only English and numbers are used, everything works normally. However, if Chinese characters are used, an error is returned, as shown below:
To solve this problem, you need to save it in UTF-8 format, usually in. add #-*-coding: UTF-8-*-to the header of the py file. In this way, python will interpret the characters in the UTF-8 encoding format and then run the command as follows:
In this case, print Chinese strings to garbled characters, which is caused by inconsistent python encoding with the console encoding. In the Windows console, gbk is used for encoding, while UTF-8 is used for code, python prints the data in the gbk encoding console according to UTF-8 encoding, but the correct Chinese characters cannot be printed.
In this regard, Python2 is used to store Unicode objects as text data types. It can be used to store and maintain Unicode data and is well integrated with existing string objects. If necessary, it provides automatic conversion. Creating a Unicode string in Python is as simple as creating a normal string. You only need to add u before the string quotation marks. In the preceding example, u "Chinese" is used ". After such processing, the printed Chinese will not be garbled.
In Python 3. x, 'xxx' and u'xxx' are unified into Unicode encoding, that is, the write prefix u is the same.
Sys. setdefaultencoding ()
Python2, you can use sys. setdefaultencoding ('utf8') to set the default encoding mode. Note that you must add reload (sys). For the reason, refer to the reference below.
1 #! /Usr/bin/env python 2 # encoding: UTF-8 3 import sys # reference the sys module. This is not the first time sys loads 4 reload (sys) # reloads sys 5 sys. setdefaultencoding ('utf8') # Call the setdefaultencoding Function
So why is it necessary to reload the function, but the function cannot be called after direct reference? Because the setdefaultencoding function is deleted after it is called by the system, it is no longer available when it is referenced through import. Therefore, the sys module must be reloaded once so that setdefaultencoding can be available, in order to modify the interpreter's current character encoding in the code.
In the Lib folder of the python installation directory, there is a file named site. in the file, you can find main () --> setencoding () --> sys. setdefaultencoding (encoding), because this site. py is automatically loaded every time the python interpreter is started, so the main function is executed every time, and the setdefaultencoding function is deleted once it comes out.