The encode of Python string and the solution to the problem of decode

Why Python Use the process will appear a variety of garbled problems, obviously the Chinese characters are displayed as "\xe4\xb8\xad\xe6\x96\x87" form?

For example, in this real life, I encountered this code:

#-*-utf-8-*-txt = "Today, the weather is fine!" Sunny. Feeling rich? "Print (Re.split (' [,.!!! ] ', TXT)

Why is the error "Unicodeencodeerror: ' ASCII ' codec can ' t encode characters in position 0-1: Ordinal not in range (128)"? Let's take a look at the problem.

The representation of a string inside Python is Unicode encoding, so in encoding conversion, it is usually necessary to use Unicode as the intermediate encoding, that is, decoding the other encoded string (decode) into Unicode first. From Unicode encoding (encode) to another encoding.

The role of Decode is to convert other encoded strings into Unicode encodings, such as Str1.decode (' gb2312 '), to convert gb2312 encoded string str1 into Unicode encoding.

The role of encode is to convert Unicode encoding into other encoded strings, such as Str2.encode (' gb2312 '), to convert Unicode encoded string str2 to gb2312 encoding.

Therefore, the transcoding must first understand, the string str is what encoding, and then decode into Unicode, and then encode into other encodings

The default encoding of the string in the code is consistent with the encoding of the code file itself .

such as: s= ' Chinese '

If it is in a UTF8 file, the string is UTF8 encoded, and if it is in a gb2312 file, it is encoded as gb2312. In this case, to encode the conversion, you need to first convert it to Unicode encoding using the Decode method, and then use the Encode method to convert it to another encoding. Typically, you create a code file by using the system default encoding when you do not specify a specific encoding method.

If the string is defined like this: S=u ' Chinese '

The encoding of the string is specified as Unicode, which is the internal encoding of Python, regardless of the encoding of the code file itself. Therefore, for this situation to do the encoding conversion, only need to use the Encode method to replace it with the specified encoding.

If a string is already Unicode, then decoding will be an error, so it is common to determine whether it is encoded as Unicode:

Isinstance (S, Unicode) # is used to determine if Unicode

Encode with a non-Unicode encoded form of STR will error

How do I get the default encoding for my system ?

#!/usr/bin/env python#coding=utf-8import sysprint sys.getdefaultencoding ()

The program is printed on the English windowsxp as: ASCII

In some Ides, the output of a string is always garbled, or even wrong, because the IDE's result output console itself cannot display the encoding of the string, rather than the problem of the program itself.

As in Ulipad, run the following code:

S=u "Chinese"
Print S

Prompt: Unicodeencodeerror: ' ASCII ' codec can ' t encode characters in position 0-1: Ordinal not in range (128). This is because Ulipad on the English windowsxp console Information Output window is ASCII encoded output (the default encoding of the English system is ASCII), and the string in the above code is Unicode encoded, so the output generated an error.

Replace the last sentence with the following: Print S.encode (' gb2312 ')

Can correctly output "Chinese" two words.

If the last sentence should read: Print S.encode (' UTF8 ')

The output: \xe4\xb8\xad\xe6\x96\x87, which is the result of the Console Information Output window UTF8 encoded strings in ASCII encoded output.

Unicode (str, ' gb2312 ') is the same as Str.decode (' gb2312 '), which converts gb2312 encoded STR to Unicode encoding

Use str.__class__ to view the encoded form of STR

Principle said a half-day, the last to a package cure all ills it:)

#!/usr/bin/env python #coding =utf-8 s= "Chinese" if isinstance (S, Unicode): #s =u "Chinese" Print s.encode (' gb2312 ') Else: #s = "Chinese" prin T S.decode (' utf-8 '). Encode (' gb2312 ')

There is also the article in the beginning of the example to solve garbled problem:

import RE txt = "Today, the weather is fine!" Sunny. Feeling rich? ". Decode (' Utf-8 ') res =re.split (' [,.!? ] ', txt) for res in Res:print res.encode (' Utf-8 '),

This article is from the "ganshizhe.bokee.com" blog, make sure to keep this source http://328538.blog.51cto.com/318538/1660309

Python string encode and decode research experience garbled problem resolution method