1.2 Python Basics-character encoding

The data in the computer is stored in binary mode, that is, only "0" and "1", binary is the data type data, it can only be converted with other binary data types, but cannot store other characters, such as: letters, special characters, etc., so people create a table, This table is a table of 10 binary numbers and characters, and this table is called ASCII(American Standard Code for Information Interchange US Information interchange) tables, Mainly used to display modern English and other Western European languages, as follows:

Now people can store data in computers, but only in English and other Western European languages, other languages of the country's data can not be stored, such as: China, Japan, Korea and other places of the language, so people have invented a new character encoding table, ASCII extension, Indexing in the ASCII table points to a new encoding table, the Extended character encoding table used in China is called "GB2312(Chinese character encoding set for information Interchange)", which is used for information exchange between Chinese character processing, Chinese character communication and other systems, and it is used in mainland China; Singapore and other places also use this code. The total income of 6,763 Chinese characters and non-Chinese characters graphic characters 682.

Because GB2312 stores too little data, a new extension table has been released, extending the existing kanji table, "GBK1.0(Chinese Internal Code specification Chinese Character Extension specification)". A total of 21,886 Chinese characters and graphic symbols, including Chinese characters (including radicals and components) 21,003, graphic symbols 883. By the year 2000, the new standard "GB18030(Information technology Chinese coded character set)", jointly issued by the Ministry of Information Industry and the National Bureau of Quality and technical supervision, was launched on March 17, 2000, and will be formally enforced as a national standard in January in 2001. China's independent development of Chinese character-based and contains a variety of Chinese minority characters (such as Tibetan, Mongolian, Dai, Yi, North Korea, Uyghur, etc.) of the super-large Chinese character set mandatory standard, which more than 70,000 of the income of Chinese characters.

can now store a lot of Chinese data, but people commonly used Chinese characters are not too many, so there will be a default in Windows in the Chinese Code table, that is, "GBK".

But now to communicate in the world's language, it is clear that ASCII is not possible, so people have invented a new encoding format, namely,Unicode(Unified Code, universal code, a single code). Unicode is created to address the limitations of traditional character encoding schemes, which set a uniform and unique binary encoding for each character in each language to meet the requirements of cross-language, cross-platform text conversion and processing. Research and development began in 1990, officially announced in 1994. Unicode is a character encoding scheme developed by international organizations that can accommodate all the words and symbols in the world. The current Unicode characters are divided into 17 groups, 0x0000 to 0X10FFFF, each group called planar (Plane), and each plane has 65,536 code bits, a total of 1,114,112. At present, however, only a few planes are used. UTF-8, UTF-16, and UTF-32 are coding schemes that convert numbers to program data.

In the Python version series, the 2.x default is that the Chinese character set is not supported, so you need to specify the encoding format in the script header:

# !/usr/bin/env python # -*-coding:utf-8-*-

However, in 3.x the default encoding format is Utf-8, that is, support Chinese characters, do not need to specify the character encoding format!

1.2 Python Basics-character encoding