When Python encounters syntaxerror: Non-ASCII character '/xd6' in file, I will teach you how to solve it.

Follow my solution to the title issues.

============================================

(1)
Use NotePad to create a file chinesetest. py. The default value is ANSI:
S = "Chinese"
Print s

Test it:
E:/project/Python/test> Python chinesetest. py

File "chinesetest. py", line 1
Syntaxerror: Non-ASCII character '/xd6' in file chinesetest. py on
Line 1, but no
Encoding
Declared
;
See http://www.python.org/peps/pep-0263.html for details

Secretly changed the file encoding into UTF-8:
E:/project/Python/test> Python chinesetest. py
File "chinesetest. py", line 1
Syntaxerror: Non-ASCII character '/xe4' in file chinesetest. py on
Line 1, but no
Encoding
Declared
;
See http://www.pytho
N.org/peps/pep-0263.html for details

No help...
Now that it provides a URL, let's take a look. After a brief look, we finally know that if the file contains non-ASCII characters, We need to specify the encoding declaration in the first or second line. Change the encoding of the chinesetest. py file to ANSI and add the encoding statement:

# Coding = GBK
S = "Chinese"
Print s

Try again:
E:/project/Python/test> Python chinesetest. py
Chinese

Normal :)
(2)
Take a look at its length:
# Coding = GBK
S = "Chinese"
Print Len (s)
Result: 4.
S is 'str' type. Therefore, a Chinese character is equivalent to two English characters, so the length is 4.
We write as follows:
# Coding = GBK
S = "Chinese"
S1 = u "Chinese"
S2 = Unicode (S, "GBK") # If the parameter is omitted, the default ASCII value of python is used for decoding.
S3 = S. Decode ("GBK") # convert STR to UNICODE: Decode. The Unicode function works the same way.
Print Len (S1)
Print Len (S2)
Print Len (S3)
Result:
2
2
2
(3)
Next let's take a look at the file processing:
Create a file named test.txt in ANSI format with the following content:
ABC Chinese
Read data using Python
# Coding = GBK
Print open ("test.txt"). Read ()
Result: ABC (Chinese)
The file format into UTF-8:
Result: ABC Juan
Obviously, decoding is required here:
# Coding = GBK
Import codecs
Print open ("test.txt"). Read (). Decode ("UTF-8 ")
Result: ABC (Chinese)
I used editplus to edit test.txt, but when I used the notepad that came with windows to edit and coexist in UTF-8 format,
Running error:
Traceback (most recent call last ):
File "chinesetest. py", line 3, in
<Module>
Print
Open ("test.txt"). Read (). Decode ("UTF-8 ")
Unicodeencodeerror: 'gbk' codec can't encode character U'/ufeff 'in
Position 0: Illegal multibyte Sequence

Originally, some software, such as Notepad, will insert three invisible characters at the beginning of the file when saving a file encoded in UTF-8 (0xef 0xbb
0xbf (BOM ).
Therefore, we need to remove these characters during reading. The codecs module in Python defines this constant:
# Coding = GBK
Import codecs
Data = open ("test.txt"). Read ()
If data [: 3] = codecs. bom_utf8:
Data = data [3:]
Print data. Decode ("UTF-8 ")
Result: ABC (Chinese)

(4) A few issues left behind
In the second part, we use the Unicode function and decode method to convert STR to Unicode. Why do the parameters of these two functions use "GBK?

The first response is that GBK (# Coding = GBK) is used in our encoding statement, but is it true?
Modify the source file:
# Coding = UTF-8
S = "Chinese"
Print Unicode (S, "UTF-8 ")
Run, error:
Traceback (most recent call last ):
File "chinesetest. py", line 3, in
<Module>
S =
Unicode (S, "UTF-8 ")
Unicodedecodeerror: 'utf8' codec can't decode bytes in position
0-1: Invalid Data
Obviously, if both sides use GBK, I keep the same UTF-8 on both sides, and it should be normal, so as not to report an error.
For a further example, if the conversion here still uses GBK:
# Coding = UTF-8
S = "Chinese"
Print Unicode (S, "GBK ")
Result: The print principle in Chinese python is as follows:
To put it simply, print in Python directly transmits the string to the operating system, so you need to decode STR to the same format as the operating system. Cp936 (almost the same as GBK) is used in windows, so GBK can be used here.

Last test:
# Coding = UTF-8
S = "Chinese"
Print Unicode (S, "cp936 ")
Result: Chinese