To understand Unicode and utf-8, try to manually convert

Source: Internet
Author: User
Manual Unicode Turn Utf-8

Colleagues asked, if the Unicode into a utf-8, there are still 0 characters. It's a bit of a question to know.
So I checked the Unicoe turn utf-8. This view of the process, or to correct the original many mistakes of the understanding.
Just log down as some of your own references.

Originally thought Unicode is Unicode,utf-8 is utf-8. In fact, Unicode is just a symbol set, just a specification, standard, it only stipulates the binary code of symbols, but does not specify how this binary code should be stored on the computer. UTF-8 is one of the implementations of Unicode, which stipulates how characters are stored, transmitted, etc. in a computer.

So how is the utf-8 stored?
Two rule:
1. For single-byte symbols, the first bit of the byte is set to 0, followed by the 7-bit Unicode code for this symbol. So for the English alphabet, the UTF-8 encoding and the ASCII code are the same
2. For n-byte symbols (n>1), the first n bits of a byte are set to 1, the n+1 bit is set to 0, and the first two digits of the following bytes are set to 10. The remaining bits that are not mentioned, all of which are Unicode codes for this symbol

Looking at the simple two rules, in the conversion time is really busy for a while.

Conversion table
Unicode Symbol Range | UTF-8 Encoding method
(hex) | (binary system)
--------------------+---------------------------------------------
0000 0000-0000 007F | 0xxxxxxx
0000 0080-0000 07FF | 110xxxxx 10xxxxxx
0000 0800-0000 FFFF | 1110xxxx 10xxxxxx 10xxxxxx
0001 0000-0010 FFFF | 11110xxx 10xxxxxx 10xxxxxx 10xxxxxx

The example given in the data is that the "strict" conversion of Chinese characters
Strict Unicode is 4E25 (100111000100101), and note that if the "strict" notebook is used for Unicode storage, use the UE to open the view in 16. Is 4E, high is low byte. If the use of other Chinese characters to convert, if with the help of the UE, you need to pay attention to this problem.

Transformation
4E25 is in the range of the third line (0000 0800-0000 FFFF), so "strict" UTF-8 encoding requires three bytes, that is, the format is "1110xxxx 10xxxxxx 10xxxxxx". Now is this x how to fill it. The first time, I Was (100111000100101) from left to right to fill in the way to this x. The result of this is
1110 1001, ... In fact, the UTF-8 code of "Yan" is "11100100 10111000 10100101". The discovery does not match. Look back at the explanation, OH. It turns out to be. The order was reversed. The first time I understood it was to fill in X from the left and the back 0. In fact, it should be from right to left to fill X, the front of the 0. Oh. is still careless.

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: info-contact@alibabacloud.com and provide relevant evidence. A staff member will contact you within 5 working days.

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.