Python character encoding, file processing, functions

One, character encoding
1. What is character encoding
The process of converting a character to a computer-recognized machine code (0,1) is called a character encoding.

2. Type of character encoding
1) Modern computers originated in the United States, the earliest born is also based on English-considered ASCII
ASCII: A bytes (byte) represents one character (English characters, all other characters on the keyboard), and 1bytes=8bit,8bit can be represented as 2 of the 8-way variation, which can represent 256 characters.
ASCII originally used only the back-tapping, 127 digits, has been fully able to represent all the characters on the keyboard (English characters/keyboard all other characters), later Latin also encoded into the ASCII table, occupy the highest level.

2) in order to satisfy Chinese, the Chinese have customized the GBK
Gbk:2bytes represents a character
In order to meet the demand, each country has customized its own code. Japanese code shift_jis, Korean code EUC-KR.

3) Countries code according to their own standards, when in multi-language mixed text, there will be conflicts, garbled display. A Unicode universal code was produced.
Unicode: Unified 2Bytes for a character, 2**16-1=65535, can represent 655,535 characters, thus compatible with the universal language.
Since the fixed use of 2 bytes represents a character, for the English text, more than one storage space, resulting in utf-8.

4) Utf-8: To judge the text, Chinese use 3Bytes, English characters only 1Bytes.

Comparison between Ps:unicode and uft-8:
Unicode: Simple rough, all characters are 2Bytes, the advantage is the character---number conversion speed, the disadvantage is the space-occupying, (memory using Unicode speed).
Utf-8: Accurate, different characters with different lengths, the advantage is to save space, the disadvantage is: character---number conversion speed is slow, because each time you need to calculate how long the character needs bytes to be able to accurately represent, (hard disk use utf-8 save space).

3, the use of character encoding
The text editor, for example, opens the read or modify in memory whenever it is opened.
When reading a file, it is the decode process: The Utf-8 formatted text in the hard disk-"decode (translated)-" Memory Unicode format.
When you save a file, it is the encode process: put the in-memory Unicode format text-"Encode (translate)-" On the hard disk in utf-8 format.

4, Python3 and 2 of the difference between the code
All strings in the Python3 are recognized as Unicode character encodings.
All strings in Python2 are the encoded result bytes bytes.

Python character encoding, file processing, functions