Python Coding issues

Source: Internet
Author: User

A summary of Python coding issues

This article analyzes and summarizes common coding problems in Python after the following article and code explain the common character encodings in detail. Because the python3.x version number and the python2.x version number differ greatly in character encoding, this article analyzes the character encoding problem in the 2.x version number with Python2.7.5.


1.Python encoded Base 1.1 STR and Unicode

There are two kinds of data models in Python that support data types such as strings, STR and Unicode, and their base classes are basestring.

For example s = "中文" , a string of type str, but u=u"中文" a string of Unicode type. Unicode is obtained by decoding a string of type Str. Unicode can also be encoded as a str type. That

STR--Decode-->unicodeunicode-----encode str

strictly speaking, Str may be called a byte string, because for UTF-8 encoded STR type "Chinese", the result of using the Len () function is 6, because the UTF-8 encoded STR type “中文” is actually "\xe4\xb8\xad\xe6\x96\x87" .

For the Unicode type U "Chinese" (actually u"\u4e2d\u6587" ), using the Len () function to get the result is 2.

1.2 Header Code Declaration

In the Python source file is assumed to be practical to non-ASCII characters, such as Chinese, then the source file must be declared in the head of the source code character encoding, the format such as the following:

#-*-Coding:utf-8-*-

This format looks more complex, in fact Python only checks #, coding, encoding and other strings, can be abbreviated to #coding:utf-8, even can write #coding:u8.


2.python2.x Common Coding Issues 2.1 header Code declaration and file Encoding issues

The file header encoding declaration determines the way in which Python parses the encoding of STR in the source code, for example, the header is Utf-8 encoded, and s="中文" Python is parsed in accordance with the Utf-8 encoding format, by repr(s) being able to see the character encoding "\xe4\xb8\xad\xe6\x96\x87" . Assuming that the code for the Head declaration is GBK encoded, Python parses s using GBK encoding. The result is "\xd6\xd0\xce\xc4" .

It should be noted that the encoding of the file itself is consistent with the file header code, otherwise it will fail.

The encoding of the file itself under Linux can be viewed under vim with a command set fenc .

Suppose the file itself is encoded as GBK. The source code file header Declaration of the Code is UTF-8, this assumes that the source code in Chinese will be problematic, because its own Chinese STR storage is based on GBK encoding. Python, while parsing str, thought it was utf-8 encoded. This will report a SyntaxError: (unicode error) ‘utf8‘ codec can‘t decode byte mistake.

2.2 Default encoding issues

Here's a look at the problem caused by the Python default encoding:

#coding: utf-8u = u "Chinese" Print repr (U) # u ' \u4e2d\u6587 ' s = "Chinese" Print repr (s) # ' \xe4\xb8\xad\xe6\x96\x87 ' U2 = S.decode ("utf -8 ") Print Repr (U2) # u ' \u4e2d\u6587 ' #s2 = U.decode (" Utf-8 ") #编码错误 #u2 = S.encode (" Utf-8 ") #解码错误

Note that the 2 lines of code that are staring out of the instance are best not to call decode directly for Unicode. STR is best not to call the Encode method directly. Because the assumption is called directly. is equivalent tou.encode(default_encoding).decode("utf-8"). Default_encoding is the default encoding used in the Unicode implementation of Python, which issys.getdefaultencoding()get the code, assuming you haven't set it up. Then the default encoding is ASCII, assuming that your Unicode itself exceeds the ASCII encoding range and will error. Similarly, if you call the Encode method directly to STR, the default is to decode STR first. S.decode (default_encoding) encode ("Utf-8"), assuming that Str itself is Chinese, and default_encoding is ASCII. Decoding will be error, resulting in the above two lines are reported separatelyUnicodeEncodeError: ‘ascii‘ codec can‘t encode characters in position...Errors andUnicodeDecodeError: ‘ascii‘ codec can‘t decode byte 0xe4 in position...error. 

The two lines of code staring out of the example above assume that the operation will be an error, assuming, of course, that either STR or Unicode is in the ASCII encoding range. There's no problem. For example s = "abc"; s.encode("utf-8") , there is no problem. Statement will return a STR that is different from the ID of S.

That assumes that there are two ways to solve the problem in instance 1, one is to understand the specified encoding, such as the following:

#coding: utf-8u = u "Chinese" Print repr (U) # u ' \u4e2d\u6587 ' s = "Chinese" Print repr (s) # ' \xe4\xb8\xad\xe6\x96\x87 ' U2 = S.decode ("utf -8 ") Print Repr (U2) # u ' \u4e2d\u6587 ' s2 = U.encode (" Utf-8 "). Decode (" Utf-8 ")  # OK                                    u2 = S.decode (" UTF8 "). Encode (" Utf-8 ")   # OK

Another way is to change the Python default encoding to the file encoding format, as seen in the following (this is only so to reload the Sys module. is because the Setdefaultencoding method was removed after Python initialization):
#coding: utf-8                                                                   import sys reload (SYS) sys.setdefaultencoding ("Utf-8") #更改默认编码为utf -8u = u "Chinese" Print repr (U) # u ' \u4e2d \u6587 ' s = "Chinese" Print repr (s) # ' \xe4\xb8\xad\xe6\x96\x87 ' U2 = S.decode ("Utf-8") print Repr (U2) # u ' \u4e2d\u6587 ' s2 = U.deco De ("Utf-8") U2 = S.encode ("Utf-8")
2.3 Read and Write file encoding

When opening a file with the Python open () method, read () reads STR, which is the encoding of the file itself. When calling write (), assuming that the reference is Unicode, you need to specify the encoding encode, assuming that the write () parameter is Unicode and that no encoding is specified, the Python default encoding is encode and then written.

#coding: utf-8                                                                   f = open ("testfile") s = F.read () f.close () print type (s) # <type ' str ' >u = S.decode ("Utf-8") #testf Ile is utf-8 encoded f = open ("Testfile", "W") F.write (U.encode ("GBK")) #以gbk编码写入, Testfile is GBK encoded f.close ()

In addition, Python's codecs module provides an open () method. Ability to specify the encoding to open the file. Use this method to open a file read that returns Unicode.

When writing, assuming that the write parameter is Unicode, then write with the encoding of the open file, assuming STR, before decoding it to Unicode with the default encoding and then writing to the encoding of the open file (it is important to note that STR is Chinese, The default encoding sys.getdefaultencoding () is ASCII, which will report decoding errors).

#coding: Gbkimport codecsf = Codecs.open (' testfile ', encoding= ' utf-8 ') u = F.read () f.close () print type (u) # <type ' Unicode ' >f = Codecs.open (' testfile ', ' a ', encoding= ' Utf-8 ') f.write (u) #写入unicode # write GBK encoded str, self-decode encoded operation s = ' Han ' Print repr (s) # ' \XBA\XBA ' # This will first decode GBK encoded STR to Unicode and then encode to UTF-8 write #f.write (s) #默认编码为ascii时. This will report a decoding error. F.close ()

3. References
    • Python character encoding detailed explanation

Copyright notice: This article blog original articles, blogs, without consent, may not be reproduced.

Python Coding issues

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: info-contact@alibabacloud.com and provide relevant evidence. A staff member will contact you within 5 working days.

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.