Solution to java Chinese garbled characters (5) ----- how java is encoded and decoded, java ----- java
In the previous blog, LZ elaborated on the transcoding process of various java channels, and elaborated on the steps during java's running for transcoding, if a problem occurs during these transcoding processes, garbled characters may occur! The following LZ describes how java performs encoding and decoding during transcoding.
Encoding & Decoding
In the previous blog, LZ elaborated on the encoding and conversion processes of the three channels. The following LZ will end the coding and decoding operations of java in those occasions, and details the intermediate process, further understand the java coding and decoding process. There are four main scenarios in java that require encoding and decoding:
1: I/O operations
2: Memory
3: Databases
4: javaWeb
The following describes the first two scenarios. In the database section, you only need to set the correct encoding format. in Java Web scenarios, you need to know too much about URL, get, POST encoding, and servlet decoding, for this reason, we will introduce LZ in the javaWeb scenario.
I/O operations
In the previous LZ raised garbled problem is nothing more than the encoding format in the transcoding process is not uniform, for example, when encoding using UTF-8, decoding using GBK, however, the most fundamental reason is that conversion from a character to a byte or byte to a character fails. In this case, the most important scenario is when I/O operations are performed. Of course, I/O operations mainly include network I/O (that is, javaWeb) and disk I/O. Network I/O.
First, let's first look at the I/O encoding operation.
InputStream is the superclass of all classes of the byte input stream, and Reader is the abstract class for reading the bytes stream. Java supports reading files by byte streams and by Bytes streams. InputStream and Reader are the superclasses of these two reading methods.
Byte
We generally use InputStream. the read () method reads only one byte (read () in the data stream at a time, which is very slow. We generally use read (byte []). save it in a byte [] array and convert it to String. When we read a file, the encoding of the bytes to be read depends on the encoding format used by the file. encoding is also involved in the conversion to String, if the encoding formats are different, the problem may occur. For example, if there is a question test.txt encoding format is UTF-8, then the data stream encoding format obtained when reading the file through byte stream is UTF-8, and if we do not specify the encoding format in the process of converting to String, by default, the system encoding format (GBK) is used for decoding. Because the two encoding formats are inconsistent, garbled characters are generated during String construction, as shown below:
File file = new File("C:\\test.txt"); InputStream input = new FileInputStream(file); StringBuffer buffer = new StringBuffer(); byte[] bytes = new byte[1024]; for(int n ; (n = input.read(bytes))!=-1 ; ){ buffer.append(new String(bytes,0,n)); } System.out.println(buffer);
Output result: Too Many Rows exist? Cm
The content in test.txt is: I am cm.
To avoid garbled characters, specify the encoding format during String construction so that the encoding formats are consistent during decoding:
buffer.append(new String(bytes,0,n,"UTF-8"));
By character
In fact, the bytes stream can be seen as a packaging stream. At the underlying layer, it uses byte streams to read bytes. Then, it uses the specified encoding method to decode the read bytes into characters. In java, Reader is a superclass that reads the volume stream. So there is no difference between reading files by byte and reading by character at the underlying level. When reading a character, each time the character is read, the byte stream reads one byte at a time.
Byte & character conversion
To convert bytes to characters, you must use InputStreamReader. The API is explained as follows: InputStreamReader is a bridge between byte stream and byte stream: it uses the specifiedcharsetRead bytes and decode them into characters. Its character set can be specified by name or explicitly specified, or it can accept the default Character Set of the platform. Each time you call a read () method in InputStreamReader, one or more bytes are read from the underlying input stream. To enable effective conversion from byte to character, you can read more byte from the bottom layer in advance, so that it exceeds the byte required for the current read operation. The API explanation is very clear. InputStreamReader still uses byte to read the file at the underlying layer. After reading the byte, it needs to parse it into characters according to a specified encoding format, if no encoding format is specified, the default encoding format is used.
String file = "C: \ test.txt"; String charset = "UTF-8"; // convert the write character into a byte stream FileOutputStream outputStream = new FileOutputStream (file ); outputStreamWriter writer = new OutputStreamWriter (outputStream, charset); try {writer. write ("I am cm");} finally {writer. close () ;}// read byte converted to character FileInputStream inputStream = new FileInputStream (file); InputStreamReader reader = new InputStreamReader (inputStream, chars Et); StringBuffer buffer = new StringBuffer (); char [] buf = new char [64]; int count = 0; try {while (count = reader. read (buf ))! =-1) {buffer. append (buf, 0, count) ;}} finally {reader. close () ;}system. out. println (buffer );
Memory
First, let's look at the following simple code.
String s = "I am cm"; byte [] bytes = s. getBytes (); String s1 = new String (bytes, "GBK"); String s2 = new String (bytes );
In this Code, we see three encoding conversion processes (one encoding and two decoding operations ). First read String. getTytes ():
public byte[] getBytes() { return StringCoding.encode(value, 0, value.length); }
Internally call the StringCoding. encode () method for operations:
static byte[] encode(char[] ca, int off, int len) { String csn = Charset.defaultCharset().name(); try { // use charset name encode() variant which provides caching. return encode(csn, ca, off, len); } catch (UnsupportedEncodingException x) { warnUnsupportedCharset(csn); } try { return encode("ISO-8859-1", ca, off, len); } catch (UnsupportedEncodingException x) { // If this code is hit during VM initialization, MessageUtils is // the only way we will be able to get any kind of error message. MessageUtils.err("ISO-8859-1 charset not available: " + x.toString()); // If we can not find ISO-8859-1 (a required encoding) then things // are seriously wrong with the installation. System.exit(1); return null; } }
The encode (char [] paramArrayOfChar, int paramInt1, int paramInt2) method first calls the default encoding format of the system. If no encoding format is specified, encoding is performed using the ISO-8859-1 encoding format by default, further details:
String csn = (charsetName == null) ? "ISO-8859-1" : charsetName;
In the same way, we can see that the new String constructor calls the StringCoding. decode () method internally:
public String(byte bytes[], int offset, int length, Charset charset) { if (charset == null) throw new NullPointerException("charset"); checkBounds(bytes, offset, length); this.value = StringCoding.decode(charset, bytes, offset, length); }
The decode method and encode process the encoding format in the same way.
For the above two cases, we only need to set a unified encoding format, which generally does not cause garbled characters.
Encoding & encoding format
First, let's look at the java coding class diagram [1].
Set the ChartSet Class Based on the specified chart, create the ChartSetEncoder object based on the ChartSet, and then call CharsetEncoder. encode encodes strings. Different encoding types are mapped to a class. The actual encoding process is completed in these classes. The following sequence diagram shows the detailed Encoding Process:
The detailed Encoding Process can be learned through the class diagram and sequence diagram of this encoding. Below will be through a simple piece of code for ISO-8859-1, GBK, UTF-8 Encoding
Public class Test02 {public static void main (String [] args) throws UnsupportedEncodingException {String string = "I am cm"; Test02.printChart (string. toCharArray (); Test02.printChart (string. getBytes ("ISO-8859-1"); Test02.printChart (string. getBytes ("GBK"); Test02.printChart (string. getBytes ("UTF-8");}/*** convert char to hexadecimal */public static void printChart (char [] chars) {for (int I = 0; I <chars. length; I ++) {System. out. print (Integer. toHexString (chars [I]) + "");} System. out. println ("");}/*** byte to hexadecimal */public static void printChart (byte [] bytes) {for (int I = 0; I <bytes. length; I ++) {String hex = Integer. toHexString (bytes [I] & 0xFF); if (hex. length () = 1) {hex = '0' + hex;} System. out. print (hex. toUpperCase () + "");} System. out. println ("") ;}------------------------- outPut: 6211 662f 20 63 6d 3F 3F 20 63 6D CE D2 CA C7 20 63 6D E6 88 91 E6 98 AF 20 63 6D
Through the program, we can see that the result of "I am cm" is:
Char []: 6211 662f 20 63 6d
ISO-8859-1: 3F 3F 20 63 6D
GBK: CE D2 CA C7 20 63 6D
UTF-8: E6 88 91 E6 98 AF 20 63 6D
The figure is as follows:
More & References
For these two scenarios, we only need to set consistent and correct encoding. Generally, there will be no garbled issues. Through the above descriptions in LZ, we should have a clear understanding of the java encoding and decoding process. In fact, the main scenario for generating garbled characters in java is in javaWeb. Therefore, the LZ post will explain the garbled characters in javaWeb.
1. Analysis and Solution of Chinese Character Problems in Java programming technology: http://www.ibm.com/?works/cn/java/java_chinese /.
----- Original from: http://cmsblogs.com /? P = 1491Please respect the author's hard work and repost the source.
----- Personal site:Http://cmsblogs.com