Read and Write Unicode text in C ++

Read and Write Unicode text in C ++

<Salute Original Author> http://librawill.blogspot.com/2008/08/cunicode_2881.html

Familiar with character types, Char, wchar_t, and tchar. The most familiar char is a single-byte character, suitable for ANSI encoding; wchar_t

It is a double-byte wide character type and applies to unicode encoding. tchar is a macro, which is defined as char and Unicode in the ANSI environment.

It is defined as wchar_t in bad environment.

How to represent a string? Right, character array, you must know that in the C ++ language, there is actually no data structure of the array, the so-called array, all

Is expressed by pointer + length. Const char *, const wchar_t *, and const tchar * can be used in different rings.

It indicates a string. Let's talk about the related macros: lpstr: Long Point string, which is equivalent to char *; lpcstr: Long Point

Const string, equivalent to const char *; lpcwstr: Long Point const wide string, equivalent to const wchar_t *;

Lpctstr: similar, equivalent to const tchar *; Do not memorize them. Remember the meaning of uppercase letters to guess their meaning.

.

A string, for example, "Beijing 2008", corresponding to the ANSI code expressed as const char * CHA = "Beijing 2008"; unicode encoding

Const wchar_t * wcha = l "Beijing 2008 ";. It is stored in binary in the memory, and the ANSI Code corresponds to 0x b1b1.

Bea9 32 30 30 38, Unicode encoding 0x1753 ac4e 3200 3000 3000.

Back to the above, Why can a struct pointer represent a string? The computer finds this pointer and can only know the first character of the string.

The string has a default Terminator '\ 0' (ANSI or ASCII is 0x00). Starting from the first character, the computer starts to look forward

If the string ends at 0x00, the computer carries a special terminator when storing the string. But pay attention to it,

The ending character 0x00 is the ending character defined by the ASCII code. What about the wide character Unicode environment? What is the Terminator? Yes

0x0000.

How do I represent a non-const string? How does the char * method dynamically define the length? You can use new to manually allocate memory space.

In addition, there is a better way, that is, string type, how to variable length, how to record length, how to store memory

Storage, all of which are left empty, and all of which are automatically managed by the C ++ standard library.

How to convert different types of strings? For example, you can define char * cha; string STR; STR = CHA; // char *

String Conversion, CHA = Str. c_str (); can be converted from string to char *; what about wchar_t wcha; wstring wstr;

? Wstr = wcha; wcha = wstr. c_str (); // can this problem be solved ?!

After talking about the string representation and type conversion, let's look at fstream, ifstream, ofstream, and file stream in streams I/O and C ++.

There are many I/O methods. The default is the ANSI simplified stream, which is specific to ANSI text.

How to read and write Unicode?

In C ++, there is really a wfsteam stream. Unfortunately, it is strange to use it. If you use wifstream to read Unicode text, the result is actually read 1.

Add 0x00 to read the next byte! For example, the text is still saved in "Beijing 2008 ".

Unicode encoding: 0x1753 ac4e 3200 3000 3000; the characters read from the memory using the wifstream are actually 0x3800

5300 ac00 4e00... What is Unicode? I don't know how to use wfstream correctly. If you know it, please leave it blank.

Zhi!

Since wftream does not work, how can we read Unicode? Here we can refer to the binary stream Reading and Writing Method, binary stream reading and writing

You must understand the data structure of the storage unit, define it as a struct, and then read data by n Bytes (n is the structure length) in binary format.

For reference, you do not need to define the structure. You can use wchar_t directly,CodeAs follows:
Ifstream fin;
Fin. Open (filename, IOS: Binary );
// Skip the Unicode text and start with two bytes 0 xfffe (called Bom, used to identify unicode encoding)
Fin. Seek (2, IOS: Beg );
While (! Fin. EOF ())
{
Wchar_t wch;
Fin. Read (char *) (& wch), 2 );
}

What should I do if I want to read data by row? Okay, the Getline (CHA, size) member function with ifstream and the string member function

Getline (FIN, STR ). Can you try Unicode? The answer is no! Why? Because the Getline function is

It is used in ANSI. It determines the Line Break Based on the ASCII code line break (0x0d) and line start mark (0x0a). If it is used in

Unicode encoding, such as the "no" character, is 0x0d4e. When the Getline function is executed here, it is considered to be a line break, so

It is invalid! So what is the binary of Unicode line breaks and line prefixes? Double Byte: 0x0d00 and 0x0a00.

The Getline function becomes invalid. What should I do? manually judge:
Ifstream fin;
Fin. Open (filename, IOS: Binary );
Size_t Index = 2;
While (! Fin. EOF ())
{
Fin. seekg (index, IOS: Beg );
Wchar_t wch;
Fin. Read (char *) (& wch), 2 );
If (wch = 0x000d) // judge the carriage return
{
Strlineansi = ws2s (wstrline );
Wstrline. Erase (0, wstrline. Size () + 1 );
Iline ++;
Index + = 4; // skip the carriage return and line start
}
Else
{
Wstrline. append (1, wch );
Index + = 2;
}
}

The aboveProgramUnicode can be read, so how can we understand Unicode? This requires

There is no simple method for conversion. The conversion between the ANSI and Unicode encoding can only be achieved through Table query. c ++ provides

Two functions, wcstombs (_ DEST, _ source, _ dsize) are converted from unicode encoding to ANSI encoding, mbstowcs (_ DEST,

_ Source, _ dsize), and vice versa. The parameter corresponds to const char *, const wchar_t *, and length. An online function is provided here.

To convert string and wstring:
STD: String ws2s (const STD: wstring & ws)
{
STD: String curlocale = setlocale (lc_all, null); // curlocale = "C ";
Setlocale (lc_all, "CHS ");
Const wchar_t * _ source = ws. c_str ();
Size_t _ dsize = 2 * ws. Size () + 1;
Char * _ DEST = new char [_ dsize];
Memset (_ DEST, 0, _ dsize );
Wcstombs (_ DEST, _ source, _ dsize );
STD: String result = _ DEST;
Delete [] _ DEST;
Setlocale (lc_all, curlocale. c_str ());
Return result;
}

STD: wstring s2ws (const STD: string & S)
{
Setlocale (lc_all, "CHS ");
Const char * _ source = S. c_str ();
Size_t _ dsize = S. Size () + 1;
Wchar_t * _ DEST = new wchar_t [_ dsize];
Wmemset (_ DEST, 0, _ dsize );
Mbstowcs (_ DEST, _ source, _ dsize );
STD: wstring result = _ DEST;
Delete [] _ DEST;
Setlocale (lc_all, "C ");
Return result;
}

Here, you can use C ++ to read Unicode text. The writing method is similar.

this article from the csdn blog, reprinted please indicate the source: http://blog.csdn.net/jtujtujtu/archive/2009/03/31/4039034.aspx