Some small knowledge of character encoding

Article content from: Http://www.liaoxuefeng.com/wiki/0014316089557264a6b348958f449949df42a6d3a2e542c000/ 001431664106267f12e9bef7ee14cf6a8776a479bdec9b9000

The writing is easy to understand OH.

For the students to start programming, a lot of times the character encoding format is very confused, write a word about the relationship between ASCII, Unicode and UTF-8:

About ASCII encoding: Since the computer was invented by the Americans, so early only 127 letters were encoded into the computer, that is, the size of letters, numbers and some symbols, the code table is called ASCII coding, such as uppercase letters A of the code is 65 , lowercase letters z The encoding is 122 .

Then, countries develop their own national language codes.

Chinese words, Chinese obviously a byte is not enough, need at least two bytes, and can not conflict with ASCII code, so, China has developed a GB2312 code to put Chinese into.

Japan put Japanese into the language Shift_JIS .

South Korea put the Korean Euc-kr in.

Countries have national standards, there will inevitably be conflicts, the result is that in multi-language mixed text, the display will be garbled.

So, Unicode was born. Unicode unifies all languages into a set of encodings, so there is no more garbled problem. The Unicode standard is also evolving, but it is most commonly used to represent a character in two bytes (4 bytes If a very remote character is used). Unicode is supported directly by modern operating systems and most programming languages.

The new problem arises again: If Unicode encoding is unified, the garbled problem disappears. However, if you write text that is basically all in English, using Unicode encoding requires more storage space than ASCII encoding, which is not cost-effective in storage and transmission.

Thus, in the spirit of saving, there has been the conversion of Unicode encoding to "Variable length encoding" UTF-8 encoding. The UTF-8 encoding encodes a Unicode character into 1-6 bytes according to a different number size, the commonly used English letter is encoded in 1 bytes, the kanji is usually 3 bytes, and only the very uncommon characters are encoded into 4-6 bytes. If the text you want to transfer contains a large number of English characters, you can save space by coding with UTF-8.

Now the computer system common character encoding working mode:

In computer memory, Unicode encoding is used uniformly, and is converted to UTF-8 encoding when it needs to be saved to the hard disk or when it needs to be transferred.

When editing with Notepad, the UTF-8 characters read from the file are converted to Unicode characters into memory, and when the edits are complete, the conversion of Unicode to UTF-8 is saved to the file.

When you browse the Web, the server converts dynamically generated Unicode content to UTF-8 and then to the browser:

Some small knowledge of character encoding