Re-recognize Unicode and UTF8 encoding

Re-recognize Unicode and UTF8 encoding

Until today, to be exact, I just knew that UTF-8 encoding and Unicode encoding are not the same, there is a difference between the embarrassing
There is a certain connection between them, look at their but do not:

UTF-8的长度是不一定的，有可能是1、2、3字节Unicode长度一定，2个字节（USC-2）UTF-8可以和Unicode互相转换

The relationship between Unicode and UTF8

Unicode (16 binary)	UTF-8 (binary)
0000-007f	0xxxxxxx
0080-07ff	110xxxxx 10xxxxxx
0800-ffff	1110xxxx 10xxxxxx 10xxxxxx

The table above has 2 meanings, the first obvious is the correspondence between Unicode and the UTF-8 character range, and one can see how Unicode and UTF-8 convert to each other:

First, the conversion from UTF-8 to Unicode.

UTF-8 encoded binary and the above 3 formats to match, matching to remove the fixed bit (the table in the non-X position), and then from right to left each 8-bit group, not enough 8 bits left not to lead, together enough 2 bytes of bits, this is the UTF-8 corresponding Unicode encoding. Take a look at some of the following examples:

The text encoding format in the above picture is UTF-8, you can see its 16 binary representation with Winhex

字符=> UTF-8  => UTF-8二进制=> 去掉固定位置凑够16位的二进制 => 16进制汉 => E6B189 => 11100110 10110001 10001001=> 01101100 01001001 => 6C49汉 => E5AD97 => 11100101 10101101 10010111=> 0101101101010111 => 5B57#下面是在chrome命令行下面运行的结果'\u6C49'"汉"'\u5B57'"字"#到这里的话，从UTF-8转换到Unicode已经是一件非常容易的事了，看看转换的伪代码读取一个字节，11100110判断该UTF-8字符的格式，属于第三种，3个字节继续读取2个字节得到 11100101 10101101 10010111按照格式去掉固定位     1011011 01010111不够16位，左边补零    01011011 01010111  => 5B57

And look at the conversion from Unicode to UTF-8.

Talk about the problem.

Say the cause of today's problem, from the front-end input many words, UTF-8 format each word up to 30 bytes, so it will be in the front-end and background verification, JavaScript is Unicode encoding, the backend program is UTF-8 encoding, now the solution is this

Front

function utf8_bytes(str){var len = 0, unicode;for(var i = 0; i < str.length; i++){unicode = str.charCodeAt(i);if(unicode < 0x0080) {++len;} else if(unicode < 0x0800) {len += 2;} else if(unicode <= 0xFFFF) {len += 3;}else {throw "characters must be USC-2!!"}}return len;}#例子utf8_bytes('asdasdas')8utf8_bytes('yrt燕睿涛')12

Background

#对于GBK字符串$len = ceil(strlen(bin2hex(iconv('GBK', 'UTF-8', $word)))/2);#对于UTF8字符串$len = ceil(strlen(bin2hex($word))/2);

5/21/2015 8:21:53 PM

The copyright belongs to the author Iforever (luluyrt@163.com) all, without the author's consent to prohibit any form of reprint, reprinted article must be in the article page obvious location to the author and the original text connection, otherwise reserve the right to pursue legal responsibility.

The above describes the re-understanding of Unicode and UTF8 encoding, including aspects of the content, I hope to be interested in PHP tutorial friends helpful.

