Java performance optimization using NIO boost performance

In software systems, because IO is slower than memory, I/O reading and writing can become a bottleneck in many situations. Increasing the I/O speed has great benefits for improving the overall performance of the system.

In Java's standard I/O, stream-based I/O implementations, InputStream and OutputStream, are provided. This stream-based implementation processes data in bytes and makes it very easy to set up various filters.

NIO is the abbreviation for new I/O and has the following characteristics:

1. provides (buffer) cache support for all primitive types;
2. Use Java.nio.charset.Charset as the character set encoding decoding solution;
3. Increase the channel object as a new primitive I/O abstraction;
4. A file access interface that supports lock and memory mapped files;
5. Selector-based Asynchronous network I/O is provided.

Unlike streaming I/O, NIO is block-based, which processes data in blocks as the base unit. In NiO, the two most important components are buffer buffers and channel channels. A buffer is a contiguous block of memory that is a transit place for NIO to read and write data. The channel represents the source or destination of the buffered data, which is used to read or write data to the buffer, and is the interface for accessing the buffer.

This paper mainly introduces the buffer and channel in NIO to improve the performance of the system.

1. NiO's buffer class and channel

In the implementation of NIO, buffer is an abstract class. The JDK creates a buffer for each Java native type,

The Channel is also used with the NIO and buffer. The channel is a two-way path that can be read and writable.

The channel can only be read and written by the application through buffer. For example, when you are reading a channel, you need to read the data into the appropriate buffer and then read it in buffer.

An example of a file copy using NiO is as follows:

    @Test    public void test() throws IOException {        //写文件通道        FileOutputStream fileOutputStream = new FileOutputStream(new File(path_copy));        FileChannel wchannel = fileOutputStream.getChannel();                //读文件通道        FileInputStream fileInputStream = new FileInputStream(new File(path));        FileChannel rChannel = fileInputStream.getChannel();                ByteBuffer byteBufferRead = ByteBuffer.allocate(1024);//从堆中分配缓冲区                while(rChannel.read(byteBufferRead)!=-1){            byteBufferRead.flip();//将Buffer从写状态切换到读状态            while(byteBufferRead.hasRemaining()){                wchannel.write(byteBufferRead);            }            byteBufferRead.clear();//为读入数据到Buffer做准备        }        wchannel.close();        rChannel.close();    }

2. The basic principle of buffer

There are three important parameters in buffer: position (position), capacity (capacity), upper limit (limit).

• Position (position): The position of the current buffer, where the data will be read or written backwards from that position.
• Capacity (capacity): The total capacity limit of the buffer.
• Upper limit: The actual capacity size of the buffer.

Then go back to the example above:

After the Bytebuffer instance is created, the location (position), capacity (capacity), and upper limit (limit) are initialized! Position is the maximum length value for both 0,capacity and limit. When read () is written to the reading file channel, the position position is moved to the next location to be entered, and the limit,capacity is not changed. The flip () operation is then performed, which moves the limit to the position position and resets the position of the position to 0. This is done to prevent the program from reading to an area that is not operating at all.

Then write the file channel to read the Bytebuffer buffer data, and the same as the write operation, the read operation will also set the location of the position to the current position. To facilitate the next reading of the data into the buffer, we call the clear () method to initialize the Position,capacity,limit.

1. Creation of buffer

The first one is created from the heap

ByteBuffer byteBufferRead = ByteBuffer.allocate(1024);

Create from an existing array

byte[] bytes = new byte[1024];ByteBuffer byteBufferRead = ByteBuffer.wrap(bytes);

2. Resetting and emptying buffers

Buffer provides some functions for resetting and emptying the Buffer state, as follows:

public final Buffer rewind()public final Buffer clear()public final Buffer flip()

rewind()The position method resets the zero and clears the flag bit (mark). The function is to prepare for extracting valid data for buffer:

out.write(buf); //从buffer读取数据写入channelbuf.rewind();//回滚bufferbuf.get(array);//将buffer的有效数据复制到数组中

clear()The position method resets the value to zero, sets the limit to the size of capacity, and clears the mark. To prepare for the re-write buffer:

buf.clear();//为读入数据到Buffer做准备ch.read(buf);

flip()The method first sets the limit to the position of the position, resets the position of the position to zero, and clears mark. Typically used for read-write conversions.

3. Flag Buffer

The flag (mark) buffer is a useful feature in data processing, and it is like a bookmark that can be used during data processing. Record your current location at any time, then return to this location at any time to speed up or simplify the data processing process. The main functions are as follows:

public final Buffer mark()public final Buffer reset()

The mark () method is used to record the current position, and the reset () method is used to return to the current position.

4. Copy Buffer

A copy buffer is a new buffer that is based on the original buffer and generates exactly the same. Here's how:

public abstract ByteBuffer duplicate()

Simply put, the new buffer generated by the copy shares the same memory data as the original buffer, and each party's data changes are mutually visible. However, the two have maintained their respective position, limit and mark. This greatly increases the flexibility of the program and provides the possibility for multi-party processing of data.

5. Buffer shards

Buffer shards are implemented using the slice () method, which creates a new child buffer, a child buffer, and a parent buffer to share data in an existing buffer.

public abstract ByteBuffer slice()

The contents of the new buffer will begin at the current position of this buffer. Changes to the contents of this buffer are visible in the new buffer, and vice versa; the position, limit, and mark of the two buffers are independent of each other. The position position of the new buffer will be zero, its capacity and limit will be the number of bytes remaining in this buffer, and its mark mark is indeterminate. When and only if this buffer is read-only, the new buffer is read-only.

6. Read-only buffer

You can use the Asreadonlybuffer () method of the buffer object to get a read-only buffer that is consistent with the current buffer and that shares the memory data. Read-only buffers are useful for data security. It is helpful to return a read-only buffer if you do not want the data to be arbitrarily modified.

public abstract ByteBuffer asReadOnlyBuffer()

7. File mapping to Memory

NIO provides a way to map files to memory for I/O operations, which can be much faster than a regular stream-based approach. This operation is mainly implemented by the Filechannel.map () method. As follows

MappedByteBuffer map = channel.map(FileChannel.MapMode.READ_WRITE, 0, 1024);

The above code maps the first 1024 bytes of a file into memory. Returns Mappedbytebuffe, which is a subclass of buffer, so you can use it as you would with Bytebuffer.

8. Processing structured data

NIO provides a way to process structured data called scattering (scattering) and aggregation (gathering).

Scattering refers to reading data into a set of data, not just one. The aggregation is the opposite.

In the JDK, the Scatteringbytechannel interface provides related operations through Gatheringbytechannel.

Below I use an example to illustrate that aggregation is written in scatter reading.

Example function: Write two sessions to a file and then read the print.

    @Test public void Test () throws IOException {String path = "D:\\test.txt";        Gather write//This is a set of data bytebuffer ByteBuffer1 = Bytebuffer.wrap ("Java is the best tool". GetBytes (Charset.forname ("UTF-8"));        Bytebuffer byteBuffer2 = Bytebuffer.wrap ("Like the Wind". GetBytes (Charset.forname ("UTF-8"));        Record data length int length1 = Bytebuffer1.limit ();        int length2 = Bytebuffer2.limit ();        Holds a reference to the Bytebuffer instance using the Bytebuffer array.        bytebuffer[] Bytebuffers = new bytebuffer[]{bytebuffer1, byteBuffer2};        Get file Write channel FileOutputStream fileoutputstream = new FileOutputStream (new file);        FileChannel channel = Fileoutputstream.getchannel ();        Start writing channel.write (bytebuffers);                Channel.close ();        Scattering reading byteBuffer1 = Bytebuffer.allocate (length1);        ByteBuffer2 = Bytebuffer.allocate (length2);        Bytebuffers = new Bytebuffer[]{bytebuffer1,bytebuffer2}; Get file read Channel FileInputStream Fileinputstream = new FileInputStream (path);        Channel = Fileinputstream.getchannel ();        Start reading Channel.read (bytebuffers);        Read System.out.println (New String (Bytebuffers[0].array (), "Utf-8"));    System.out.println (New String (Bytebuffers[1].array (), "Utf-8")); }

When execution is complete, we open the Test.txt file and see: Java is the best tool like the wind

and print it out on the console:

Java是最好的工具像风一样

9. Direct Memory Access

The NIO Buffer also provides a class----directbytebuffer that can directly access the system's physical memory.

Directbytebuffer inherits from Bytebuffer, but differs from normal buffer. The normal bytebuffer still allocates space on the JVM heap, and its maximum memory is limited by the maximum heap. The directbytebuffer is allocated directly in physical memory and does not occupy heap space. Moreover, Directbytebuffer is a way of getting closer to the bottom of the system, so it's faster than normal bytebuffer.

Easy to use, just replace the bytebuffer.allocate (1024) with Bytebuffer.allocatedirect (1024). The source of the method is

    public static ByteBuffer allocateDirect(int capacity) {        return new DirectByteBuffer(capacity);    }

It is necessary to note that using the parameter-xx:maxdirectmemorysize=10m can specify a maximum size of directbytebuffer of 10M.

Directbytebuffer is faster to read and write than normal buffer, but it is slower to create and destroy than normal buffer. However, if Directbytebuffer can be reused, it can significantly improve system performance in the case of frequent read and write.

3. Comparison with traditional I/O

Using the traditional I/O implementation of the initial file copy example, the code is as follows:

    @Test    public void test6() throws IOException {        //缓冲输出流        BufferedOutputStream bufferedOutputStream = new BufferedOutputStream(new FileOutputStream(new File(path_copy)));        //缓冲输入流        BufferedInputStream bufferedInputStream = new BufferedInputStream(new FileInputStream(new File(path)));        byte[] bytes = new byte[1024];        while (bufferedInputStream.read(bytes) != -1) {            bufferedOutputStream.write(bytes);        }        bufferedInputStream.close();        bufferedOutputStream.close();    }

It is important to note that although using Bytebuffer to read and write files is much faster than stream, it is not enough to indicate that there is such a big gap between the two. This is because Bytebuffer is a one-time read of the file into memory for subsequent processing, and stream mode is the edge-read file edge processing data (although the use of buffer component Bufferedinputstream), which is the cause of the difference in performance. Even so, the advantages of using NIO cannot be concealed. Using NiO instead of traditional I/O operations, optimization of the overall performance of the system should have an immediate effect.

Attached: What does the application's byte and file size KB really matter?

Here, we can see in the computer how much the file size is KB,MB and so on, and we are using the basic data type byte in the system to manipulate the data. So what is the relationship between byte and KB, it is necessary to understand. Take 1KB files for example:

1Byte = 8Bit1KB = 1024Bit1KB = 128Byte

When we do byte[] bytes = new byte[1024] this, it is equivalent to opening up 8KB of memory space.

Java performance optimization using NIO boost performance