Thoroughly decrypts C ++ wide characters

From character to integer

Char is an integer. The meaning of this sentence is that all the characters that char can represent are integers in C/C ++. Well, next, many articles will give a typical example. For example, the value of 'A' is 0x61. Is this true? If you carefully read the original description of C and C ++ by K & R and BS, you will immediately refute that 0x61 is only the ASCII value of 'A, there is no rule that the char value of C/C ++ must correspond to ASCII. C/C ++ does not even specify the number of char digits. It only specifies that sizeof (char) is equal to 1.
Of course, in most cases, char is 8-bit, and the value in the ASCII range corresponds to ASCII.

Locale)

"Translate 'A' into an integer of 0x61" and "associate the encoding in the ASCII range with the integer of char", similar to this provision, it is developed by a specific system and compiler. C/C ++ has a specific term to describe the set of such regulations: locale. It can also be translated into "on-site "). And Translation-that is, Code Conversion (codecvt) is only one of this set, and is defined as a policy (FACET) in C ++. It can also be translated into "face-to-face ")

C/C ++ compilation Policy

"Localization policy set" is a good concept. Unfortunately, at the character and string level, c/C ++ is not used (the locale of C ++ usually only affects the stream). C/C ++ uses a simpler policy: Hard encoding.
In short, the characters (strings) are represented in program files (executable files, non-source files), which are consistent with those in the memory during program execution. Consider two situations:
A. Char C = 0x61;
B. Char c = 'a ';
In case a, the compiler can directly recognize the C as an integer, but in case B, the compiler must translate 'A' into an integer. The compiler policy is also very simple, that is, to directly read the encoding values of characters (strings) in the source file. For example:
Const char * s = "Chinese ABC ";
This string is encoded in gb2312 (Windows 936), which is the default Chinese system source file in windows:
0xd6 0xd0 0xce 0xc4 0x61 0x62 0x63
In the UTF-8, that is, the encoding in the Linux default system source file is:
0xe4 0xb8 0xad 0xe6 0x96 0x87 0x61 0x62 0x63
Generally, the compiler will faithfully assign values to S for the source file encoding, exceptional circumstances such as VC will be smart to convert most other types of encoded strings into gb2312 (except for survivors like UTF-8 without signature ).
During the execution of the program, s maintains this encoding and does not perform other conversions.

Wchar_t
As char does not specify the size, wchar_t does not have a standard limit. The standard only requires that a wchar_t can represent any character that the system can recognize. in Win32, wchar_t is 16 characters; linux is 32-bit. Wchar_t also does not specify encoding, because Unicode is explained later, so here just mention, in Win32, wchar_t encoding is UCS-2BE; linux is a UTF-32BE (equivalent to a UCS-4BE), but simply put, within 16 bits, the three encoding values of one character are the same. Therefore:
Const wchar_t * Ws = l "Chinese ABC ";
Encoding:
0x4e2d 0x6587 0x0061 0x0062 0x0063 // Win32, 16-bit
0x00004e2d 0x00006587 0x00000061 0x00000062 0x00000063 // Linux, 32-bit
The uppercase l tells the compiler that this is a wide string. Therefore, at this time, the compiler must translate according to locale.
For example, in a Windows environment, the compiler's translation policy is gb2312 to UCS-2BE; In a Linux environment, the policy is UTF-8 to UTF-32BE.
At this time, it is required that the source file encoding be consistent with the compiler's localization policy for centralized code translation. For example, VC can only read the source code of gb2312 (here is an exception, VC is too clever, will be a lot of other code automatically converted to gb2312 at compilation), and GCC can only read the source code of the UTF-8 (here there is an embarrassment, mingw run Win32, so only the gb2312 system recognized; while mingw is written with GCC, so I only recognize the UTF-8, so the result is that the width of mingw is discarded ).
The wide character (string) is translated by the compiler or hard-coded into the program file.