Article 2: Java character encoding Series II: Unicode, ISO-8859-1, GBK, UTF-8 encoding and mutual conversion
1. Function IntroductionIn Java, a string is encoded in Unicode. Each character occupies two bytes. The two major functions related to encoding are: 1) parse the string into a byte array using the specified encoding set, unicode-> charsetname conversion public byte [] getbytes (string charsetname) throws unsupportedencodingexception 2) the byte array is constructed into a string using the specified encoding set, charsetname-> Unicode conversion Public String (byte [] bytes, string charsetname) throws unsupportedencodingexception
2. Direct conversion between Unicode and various encodingsThe following uses the encoding conversion of the Chinese character string "A Chinese" as an example to understand the conversion between various encodings. 1) Unicode and GBK test results are as follows, it is reversible, that is, the byte can be converted back to the string-GBK> bytearray:/u0061/u4e2d/u6587 (a Chinese)-> 0x61 0xd6 0xd0 0xce 0xc4bytearray-gbk> string: 0x61 0xd6 0xd0 0xce 0xc4->/u0061/u4e2d/u6587 (a Chinese) 2) Unicode and UTF-8 test results are as follows, each Chinese character is converted to three bytes, and is reversible, that is, the byte can be converted back to the string String-UTF-8> bytearray:/u0061/u4e2d/u6587 (a Chinese) -> 0x61 0xe4 0xb8 0xad 0xe6% 0x96 0x87bytearray-utf-8> string: 0x61 0xe4 0xb8 0x Ad 0xe6% 0x96 0x87->/u0061/u4e2d/u6587 (a Chinese) 3) Unicode and ISO-8859-1 test results are as follows: conversion failure, reversible, that is, the byte can no longer be converted back to the string String-ISO-8859-1> bytearray:/u0061/u4e2d/u6587 (a Chinese)-> 0x61 0x3f 0x3fbytearray-iso-8859-1> string: 0x61 0x3f 0x3f->/u0061/u003f/u003f (??)
3. Cross conversion between Unicode and each EncodingIn the preceding direct conversion, the byte array generated by the string (UNICODE) uses the correct encoding set when constructing the return string, what if the correct encoding set is used? Will it be constructed correctly? Can I recover it if it cannot be correctly constructed? Will the information be lost? Next let's take a look at this situation. This part shows that in some cases, although we finally correctly display the results, the conversion is still incorrect. 1) can correctly display the intermediate incorrect conversion we know that string-GBK> bytearray-GBK> string is correct, but what if we use string-GBK> ByteArray-ISO-8859-1> string? Test results: String-GBK> ByteArray-ISO-8859-1> string:/u0061/u4e2d/u6587 (a Chinese) -> 0x61 0xd6 0xd0 0xce 0xc4->/u0061/u00d6/u00d0/u00ce/u00c4 (????) In this case, the string we get is? Garbled "????", But by continuing the conversion we can still restore the correct string "A Chinese", the process is as follows: String-GBK> ByteArray-ISO-8859-1> String-ISO-8859-1> bytearray-GBK> string correspondence: /u0061/u4e2d/u6587 (a Chinese)-> 0x61 0xd6 0xd0 0xce 0xc4->/u0061/u00d6/u00d0/u00ce/u00c4 (????) -> 0x61 0xd6 0xd0 0xce 0xc4->/u0061/u4e2d/u6587 (a Chinese) that is, when we construct a string for the first time, we use the wrong encoding set to get the wrong garbled code, but we add an error in the error, use the wrong encoding set to get the byte array, and then construct it with the correct encoding set, the correct string is restored. In this case, it is "Incorrect conversion in the middle that can be correctly displayed ". This often happens when submitting data processing on JSP pages. In addition, the intermediate incorrect conversions that are correctly displayed are: String-UTF-8> ByteArray-ISO-8859-1> String-ISO-8859-1> ByteArray-UTF-8> string and String-UTF-8> bytearray-GBK> string-GBK> ByteArray-UTF-8> string: /u0061/u4e2d/u6587 (a Chinese)-> 0x61 0xe4 0xb8 0xad 0xe6% 0x96 0x87->/u0061/u6d93/ue15f/u6783 (A Juan) -> 0x61 0xe4 0xb8 0xad 0xe6% 0x96 0x87->/u0061/u4e2d/u6587 (a Chinese)
4. error diagnosis reference during encoding1) a Chinese character corresponds to a question mark in the ISO-8859-1 from the string to obtain the byte array, because a unicode converted into a byte, when encountered unknown Unicode, convert to 0x3f, in this way, no matter which encoding is used for construction,? Garbled. 2) A Chinese Character corresponds to two question marks when getting a byte array from the string through GBK, because a Unicode is converted to two bytes, if at this time with the ISO-8859-1 or using the UTF-8 to construct the string will have two question marks. If it is constructed through a ISO-8859-1, it can be restored by adding an error through the above-mentioned error (that is, it is resolved from the ISO-8859-1 and constructed with GBK ); if it is constructed by UTF-8, it will produce Unicode Character "/ufffd", can not be restored, if it is through the String-UTF-8> bytearray-GBK> string, there will be a complex code, such as a small copy 3) A Chinese Character pair should contain three question marks when obtaining the byte array from the string through the UTF-8, because a Unicode is converted to three bytes, if at this time using the ISO-8859-1 to construct the string will appear three question marks; if you use GBK to construct a string, a complex code will appear, such as a Juan.
