Python learning-encoding conversion, python encoding conversion

Python learning-encoding conversion, python encoding conversion
Unicode

Note: Unicode is only a collection of symbols. It specifies the binary code of the symbols, but does not specify how to store the binary code.

The Unicode encoding method refers to the Unicode Binary Code directly stored in the symbol.

UTF-8

UTF-8 is the most widely used Unicode implementation method on the Internet.

UTF-8 is a variable-length encoding method, which uses 1-4 bytes to represent a symbol, depending on different symbols select different length bytes to represent.

UTF-8 coding rules are very simple, only two:

1) for a single-byte symbol, the first byte is set to 0, and the last seven digits are the unicode code of this symbol. Therefore, for English letters, the UTF-8 encoding and ASCII code are the same.

2) for the n-byte symbol (n> 1), the first n bits of the first byte are set to 1, and the n + 1 bits are set to 0, the first two bytes are set to 10. The remaining unmentioned binary bits are all unicode codes of this symbol.

Unicode symbol range (hexadecimal)	UTF-8 coding method (Binary)
0000 0000-0000 007F	0 xxxxxxx
0000 0080-0000 07FF	110 xxxxx 10 xxxxxx
0000 0800-0000 FFFF	1110 xxxx 10 xxxxxx 10 xxxxxx
0001 0000-0010 FFFF	11110xxx 10 xxxxxx 10 xxxxxx 10 xxxxxx

According to the above table, the interpretation of UTF-8 encoding is very simple. If the first byte is 0, the byte is a single character. If the first byte is 1, the number of consecutive 1 represents the number of bytes occupied by the current character.

Unicode to UTF-8 conversion method:

In Windows, the easiest way is to use NotePad to open the document and select the encoding method to save it.

GBK

GBK is compatible with GB2312 after expansion based on the National Standard GB2312. The GBK text encoding is expressed in double bytes, that is, both Chinese and English characters are expressed in Double Bytes. To distinguish Chinese characters, set the highest bit to 1. GBK contains all Chinese characters. It is a national code and has poor universality than UTF8. However, the database occupied by UTF8 is larger than that occupied by GBD. Both GBK and GB2312 and UTF8 can be converted to each other only through Unicode encoding.

Encoding conversion

[In python 2.7]

Temp = "James" # utf-8temp_unicode = temp. decode ('utf-8') # specify the original encoding temp_gbk = temp_unicode.encode ('gbk') for decoding # specify the encoding print (temp_gbk) for encoding)

[In python 3.5] -- automatic conversion; removed the unicode type of python

Temp = "James" temp_gbk = temp. encode ('gbk') print (temp_gbk)