[C # Reference] character encoding

"Every software developer must have at least the knowledge of Unicode and character sets (no exceptions)," says Microsoft's fart Joel (the bull who wrote "Joel says Software"), and I'm often bothered by a lot of questions about character set conversions, so this is a decision to make clear.

Learn about character encoding in this blog, starting with two programs:

classtestdatagenerator{ Public Static voidCreatenewtestdatafile (stringFileName,intrecord_length) {        using(FileStream fs =file.create (FileName)) {StringBuilder SB=NewStringBuilder ();  for(inti =0; i < record_length; i++) {sb. Append ('of the'); }// for            byte[] content =Encoding.Unicode.GetBytes (sb.)            ToString ()); Fs. Write (Content,0, content.        Length); }//using}//Createnewtestdatafile ()}

In the code above:

Encoding.Unicode.GetBytes (sb.) ToString ());

The meaning of this sentence is to encode a string into a UTF-16 byte stream.

Call this function to generate a test file containing 100 characters:

Testdatagenerator.createnewtestdatafile ("Test.txt", 100);

You can see that the file size is 200 bytes. The reason is that UTF-16 uses 2 bytes to store characters including Chinese characters and ASCII codes .

To read a Unicode string:

FileStream fs = new System.IO.FileStream ("Test.txt", FileMode.Open, FileAccess.Read); byte[] blob = new Byte[100];fs. Read (BLOB, 0, +); fs. Flush (); string strUtf16 = Encoding.Unicode.GetString (BLOB); string StrUtf8 = Encoding.UTF8.GetString (BLOB);

Visible from the Watch window, the strong conversion of strings to UTF-8 form is garbled, because the UTF-8 standard uses 3 bytes to store characters such as kanji, rather than the 2 bytes of a UTF-16.

The following characters are stored using UTF-8 encoding:

byte[] content = Encoding.UTF8.GetBytes (sb.) ToString ()); fs. Write (content, 0, content. Length);

You can then see that the file size is 300 bytes:

Similarly, it is not possible to use UTF-16 encoding to read files stored in UTF-8 encoding mode.

Explain several concepts to the above code:

C # Class Encoding Encoding.unicode property: Gets the encoding in UTF-16 format using Little-endian byte order.

C # class Encoding's Encoding.ascii property: Gets the encoding of the ASCII (7-bit) character set.

C # Class Encoding Encoding.UTF8 property: Gets the encoding of the UTF-8 format.

[C # Reference] character encoding