Character Set encoding ANSI and Unicode

Character Set encoding ANSI and Unicode
Encoding refers to the storage and interpretation of languages in different countries in computers.
ANSI and ASCII

N initially, there was only one character set on the Internet-the ansi ascii character set (American Standard Code for information interchange, "American Standard Code for information exchange), which represented a character using 7 bits, it represents a total of 128 characters. Later, IBM expanded it on this basis and represented a single character with 8 bits. It can represent a total of 256 characters, making full use of the maximum information one byte can express.
Nansi Character Set: ASCII character set, and the derived and compatible character set, for example, gb2312. The formal name is MBCS (Multi-byte chactacter system, multi-Byte Character System ), it is also known as the ANSI character set.

Unicode, utf8, UTF16

N because each language has its own character set, there are too many character sets at last, and it is inconvenient to convert character sets frequently during international exchanges. Therefore, a Unicode Character Set is generated. It uses 16 bits (two bytes) to represent a character, which can represent 65536 characters in total.
The n standard Unicode is known as the UTF-16 (UTF: UCS Transformation Format ). Later, in order to enable the dual-byte Unicode to be correctly transmitted in the existing single-byte processing system, a UTF-8 emerged and Unicode was encoded in a way similar to MBCS. (UNICODE character sets have multiple encoding formats)
For example, the Unicode Standard UTF-16 (big endian) of the Connected Word is: de 8f 1A 90
And its UTF-8 code: E8 BF 9e E9 80 9A

N when a software opens a text, the first thing it needs to do is to decide which character set is used and which encoding is saved. The Software generally uses three methods to determine the character set and encoding of the text:
check the file header ID and prompt the user to select, based on certain rules, we guess
the most standard way is to detect the first few bytes of the text, the first byte charset/encoding, as shown in the following table:
ef bb bf UTF-8
Fe FF UTF-16/UCS-2, little endian
FF Fe UTF-16/UCS-2, big endian
FF Fe 00 UTF-32/UCS-4, little endian.
00 00 Fe FF UTF-32/UCS-4, big-Endian.