Detailed explanation of C ++ natural language encoding Series 1 (character encoding basics)

Three encoding types

The first encoding type is single-byte character set or SBCS ). In this encoding mode, all characters are represented in only one byte. ASCII is SBCS. The value 0 in one byte indicates the end of The SBCS string.

The second encoding mode is the multi-byte character set or MBCS ). An MBCS encoding contains some characters long in one byte, while others are larger than the length of one byte. In Windows, MBCS contains two character types: single-byte characters and double-byte characters ). Because most of the Multi-byte characters used in Windows are two bytes long, MBCS is often replaced by DBCS.

In DBCS encoding mode, some specific values are reserved to indicate that they are part of dubyte characters. For example, in Shift-JIS encoding (a common Japanese encoding mode), the value between 0x81-0x9f and 0xe0-oxfc indicates "This Is A dubyte character, and the next byte is part of this character. "Such values are called" leading bytes ", and they are all greater than 0x7f. The Bytes following a leading byte are called "trail byte ". In DBCS, the trail byte can be any value other than 0. Like SBCS, the ending mark of the DBCS string is also 0 represented by a single byte.

The third encoding mode is Unicode. Unicode is a two-byte encoding mode for all characters. Unicode characters are also called wide characters because they are wider than single-byte characters (more storage space is used ). Note that Unicode cannot be considered as MBCS. The unique feature of MBCS is that its characters are encoded in bytes of different lengths. A Unicode string uses 0 in two bytes as its end flag.

The single-byte character contains the Latin alphabet, accented characters, and ASCII standard and graphic characters defined by the DOS operating system. Dubyte characters are used to represent the languages of East Asia and the Middle East. Unicode is used in the COM and Windows NT operating systems.

You must be familiar with single-byte characters. When you use char, you are processing single-byte characters. Double byte characters are also operated using the char type (this is one of the many strange points we will see about double byte characters ). Unicode characters are represented by wchar_t. Unicode characters and string constants are expressed by the prefix L. For example:

Wchar_t wch = L '1'; // 2 bytes, 0x0031
Wchar_t * wsz = L "Hello"; // 12 bytes, 6 wide characters