Unicode to UTF-8 encoding

I. Unicode Introduction

Unicode can be encoded using any of the following character encoding schemes:

• UTF-8

• UTF-16

• UTF-32

A Unicode encoded file has a flag, as shown in the following code:

Unicode file header ID
Byte-order mark Description
Ef bb bf UTF-8
Ff fe UTF-16 aka UCS-2, little endian
Fe ff UTF-16 aka UCS-2, big endian
00 00 FF Fe UTF-32 aka UCS-4, little endian.
00 00 Fe FF UTF-32 aka UCS-4, big-Endian.

A UTF-8 is a form of variable-length Unicode that transparently retains the ASCII character code value. This form is used as file code in the Solaris Unicode language environment.

UTF-16 is a form of Unicode 16-bit encoding. In the UTF-16, up to 65,535 characters are encoded as a single 16-bit value. The ing between 65,535 and 1,114,111 characters is encoded as the correct 16-bit value (proxy ).

UTF-32 is a fixed-length 21-bit encoding form of Unicode, typically used in 32-bit containers or data types. This form is used as process code (wide character code) in the Solaris Unicode language environment ).

We generally say that unicode encoding generally refers to UTF-16 encoding.

Ii. UTF-8 Introduction

UTF-8 is the most widely used unicode implementation method on the Internet. Other implementations also include UTF-16 and UTF-32, but are basically not needed on the Internet.Repeat, the relationship here is that UTF-8 is one of the Unicode implementation methods.

Unicode is a fixed length, no matter what is stored in two bytes, And the UTF-8 is variable, for Chinese characters, Unicode occupies 1 byte less than the UTF-8 occupies 1 byte. Unicode is double byte, while Chinese characters in the UTF-8 are three bytes.
Net soul soldier Co., http://xdotnet.cnblogs.com.
In theory, UTF-8 encoding characters can be up to 6 bytes long, but 16-bit BMP (Basic Multilingual Plane) characters can be up to 3 bytes long. Let's take a look at the UTF-8 encoding table:

U-00000000-U-0000007F: 0 xxxxxxx
U-00000080-U-000007FF: 110 xxxxx 10 xxxxxx
U-00000800-U-0000FFFF: 1110 xxxx 10 xxxxxx 10 xxxxxx
U-00010000-U-001FFFFF: 11110xxx 10 xxxxxx 10 xxxxxx 10 xxxxxx
U-00200000-U-03FFFFFF: 111110xx 10 xxxxxx 10 xxxxxx 10 xxxxxx
U-04000000-U-7FFFFFFF: 1111110x 10 xxxxxx 10 xxxxxx 10 xxxxxx 10 xxxxxx

Iii. Implementation

 

void UnicodeToUTF_8(char* pText, char* pOut){    char* pchar = (char *) pText;    pOut[0] = (0xE0 | ((pchar[1] & 0xF0) >> 4));    pOut[1] = (0x80 | ((pchar[1] & 0x0F) << 2)) + ((pchar[0] & 0xC0) >> 6);    pOut[2] = (0x80 | (pchar[0] & 0x3F));    return;}

This part has not been written yet. It is to be continued ....
 

References:

Http://www.ruanyifeng.com/blog/2007/10/ascii_unicode_and_utf-8.html

Http://www.cnblogs.com/xdotnet/archive/2007/11/23/unicode_and_utf8.html

Http://hi.baidu.com/lxf_120/item/d855536ea21e1135ad3e8328