Java character encoding and decoding

1, character encoding development process

　　①, ASCII Code

Because computers only know numbers, all of the data we have in the computer is represented by numbers, because English characters are limited, so the highest bit of the specified byte is 0, and each byte is represented by a number between 0-127. For example A corresponds to 65,a 97. This is the American Standard Information Interchange code, ASCII

String str = new String ("Aa"); byte[] Strascii = str.getbytes ("ASCII"); System.out.println (arrays.tostring (STRASCII));//[65, 97]

　　

②, GB2312 yards

With the popularization of computers in the world, many countries and regions have introduced their characters into computers, such as Chinese characters. At this point a word energy-saving representation of the number range is too small to contain all Chinese characters. Then it is required to use two bytes to represent a Chinese character.

Rule: The encoding of the original ASCII character remains unchanged and is still represented by a byte, in order to differentiate between a Chinese character and two ASCII characters. The highest bit of each byte of a Chinese character is defined as 1 (that is, the binary is a negative number), which is the GB2312 encoding

String str = new string ("AA handsome Pot"); byte[] Strascii = str.getbytes ("GB2312"); System.out.println (arrays.tostring (STRASCII));//[65, 97,-53,-89,-71,-8]

　　

③, GBK

Because of the Chinese characters too much, on the basis of GB2312 added more Chinese characters, this encoding is GBK

Question: If only in China, then everyone knows Chinese characters, but if it is another country, and the country's code table is not included in the Chinese characters. So the computer is garbled or other characters when it's displayed.

Workaround: In order to address the impact of localized character encoding in each country, all characters in the world are encoded uniformly---Unicode encoding

At this time a character is displayed in any place around the world is fixed, such as the Chinese elder brother, in any place is represented by hexadecimal 54E5.

Unicode character encoding occupies two bytes

④, UTF-8

is a variable-length character encoding for Unicode, also known as the Universal Code, which is one of the ways Unicode is implemented. The first byte in the encoding is still compatible with ASCII, which makes it possible for the original software to process ASCII characters to continue using without or requiring only a few modifications. As a result, it is gradually becoming the preferred encoding for e-mail, Web pages, and other applications that store or transmit text. Internet Engineering Working Group (IETF) requires that all Internet protocols must support UTF-8 encoding

String str = new string ("AA handsome Pot"); byte[] Strascii = str.getbytes ("UTF-8"); System.out.println (arrays.tostring (STRASCII));//[65, 97,-27,-72,-123,-23,-108,-123]

　　

Store letters, numbers: No matter what character Set occupies 1 bytes

Storing Chinese characters: the GBK family occupies 2 bytes. UTF-8 occupies 3 bytes

You cannot use a single-byte character set (ascii/iso-8859-1) to store Chinese

2. Encoding and decoding of characters

　　Information is transmitted in a computer network in the form of bytes. So how do you turn into bytes? This is the process of coding. So the computer received this code, how to let users know it? It is necessary to convert the bytes into a character-recognized string, which is the process of decoding.

Encoding: Converting a string to a byte array

Decoding: Converting a byte array to a string

Note: ①, encoding format and decoding format must be consistent, otherwise garbled

String str = new string ("AA hot Pot");//code operation byte[] Strbyte = str.getbytes ("GBK"); System.out.println (arrays.tostring (Strbyte));//[65, 97,-53,-89,-71, -8]//decode operation  // Note that the encoded character set and the decoded character set format must be consistent (either its extended character set or it can), otherwise it will be garbled//first: The encoding format is GBK, the decoding format is iso-8859-1  then it will be garbled string str2 = new String (Strbyte, " Iso-8859-1 "); System.out.println (STR2); Aa?§?? The second type: encoding and decoding format consistent string str3 = new String (Strbyte, "GBK"); System.out.println (STR3); AA Cool Pot

②, sometimes encoded as and decoding format consistent, but still garbled, this is because the data in the transmission process through the server, and this server may be written by foreigners, then the data will be converted to another character format, then you can directly to the format you want is garbled.

Workaround: First obtain the data after the server to restore the code, and then in the decoding

String str = new string ("AA hot Pot");//code operation byte[] Strbyte = str.getbytes ("UTF-8"); System.out.println (arrays.tostring (Strbyte));//[65, 97,-27,-72,-123,-23,-108, -123]//in the middle of the transfer of the server, the encoding format turned into Iso-8859-1string str2 = new String (Strbyte, "iso-8859-1");//decoding Operation  , if decoded directly at this time, then the string str3 = new string is garbled ( Str2.getbytes (), "UTF-8"); System.out.println (STR3); Aa??????? For the above garbled, we must first restore the server before the encoding format, and then in the decoding. Then it will not garbled byte[] StrByte2 = str2.getbytes ("iso-8859-1"); String STR4 = new String (StrByte2, "UTF-8"); System.out.println (STR4); AA Cool Pot

　　

Java character encoding and decoding