Java IO, NIO, Aio detailed

Overview

Before we learn the IO stream of Java, we need to know a few key words

• Synchronous and Asynchronous (Synchronous/asynchronous): synchronization is a reliable and orderly operation mechanism, and when we perform synchronous operations, the subsequent task is to wait for the current call to return before the next step is performed, whereas async is the opposite Other tasks do not need to wait for the current call to return, usually relying on events, callbacks and other mechanisms to achieve the inter-task order relationship
• Blocking and non-blocking: When a blocking operation occurs, the current thread is blocked and unable to perform other tasks, only if the condition is ready to continue, such as when a new connection is serversocket, or when the data is read, the write operation is complete, and not the blocking Whether the IO operation is finished or not, the operation will continue in the background.

The concept of synchronous and asynchronous: actual I/O operations

Synchronization is required to wait after a user thread initiates an I/O request or to poll for kernel I/O operations to complete before continuing execution

Async is a user thread that continues to execute after initiating an I/O request, notifies the user thread when the kernel I/O operation completes, or invokes a callback function registered by the user thread

Blocking and non-blocking concepts: initiating I/O requests

Blocking means that I/O operations need to be completely completed before they can be returned to user space

Non-blocking refers to the return of a status value immediately after an I/O operation is called, without the need for I/O operations to complete completely

Overview of BIO, NIO, AIO

First, the traditional java.io package, which is based on the flow model, provides some of the IO features we know best, such as File abstraction, input and output streams, and so on. The interaction mode is synchronous, blocking, that is, when reading the input stream or writing to the output stream, the thread will block there until the read and write actions are complete, and the calls between them are reliable linear order.

The advantage of the java.io package is that the code is simple and intuitive, and the disadvantage is that IO efficiency and extensibility have limitations, which can easily become the bottleneck of application performance.

In many cases, people also classify some of the network APIs provided below java.net, such as sockets, ServerSocket, and HttpURLConnection, into the synchronous blocking IO class library because network traffic is also an IO behavior.

Second, the NIO framework (Java.nio package), introduced in Java 1.4, provides new abstractions such as Channel, Selector, and Buffer, which can be used to build multiplexed, synchronous, non-blocking IO programs, while providing high-performance data manipulation approaches that are closer to the underlying operating system.

Thirdly, in Java 7, NIO has been further improved, that is, NiO 2, which introduces asynchronous nonblocking Io, and many people call it AIO (asynchronous IO). Asynchronous IO operations are based on the event and callback mechanism, which can be understood simply as the application operation is returned directly without blocking it, and when background processing is complete, the operating system notifies the appropriate thread to follow up.

I. IO stream (synchronization, blocking) 1, overview

IO flow is simply the input and output stream, the IO stream is mainly used to deal with the data transfer between devices, Java IO for the operation of the database is through the stream implementation, and Java for the operation of the flow of objects are in the IO package.

2. Classification

By operation data divided into: byte stream (Reader, Writer) and character stream (InputStream, OutputStream)

By direction of flow: input stream (Reader, inputstream) and output stream (Writer, OutputStream)

3. Overview of Character streams

Only for working with text data

The most common manifestation of data is a file, and the subclasses of the character stream used to manipulate the file are generally FileReader and filewriter

Notes on character stream read and write files:

• Write file must be flush () flush
• Run out of flow remember to close the stream
• To throw an IO exception using a Stream object
• When defining a file path, you can use "/" or "\"
• When creating a file, a file with the same name will be overwritten if the directory
• When reading a file, you must ensure that the file already exists or throw an exception

Buffer of character stream

• The presence of buffers is to improve the efficiency of the flow.
• A constructor that needs to be passed as an argument to a buffer by an improved efficiency stream
• An array is encapsulated in the buffer, and once the data is stored

4. Overview of Byte stream

Used to process media data

Notes on byte stream reading and writing files:

• The basic operation of a byte stream and a character stream is the same, but a byte stream is required to manipulate the media stream
• Byte stream can be used to manipulate media files (media files are also stored in bytes) because they operate on bytes
• Input stream (inputstream), output stream (OutputStream)
• A byte stream operation can be done without flushing the flow
• InputStream Unique method: Int available () (returns the number of bytes in the file)

Buffer of Byte stream
The byte stream buffer is the same as the character stream buffer and is also designed to improve efficiency

5. Java Scanner class

Java 5 Adds the Java.util.Scanner class, which is a new utility for scanning input text

Understanding of Nextint (), Next (), nextline ()

Nextint (): Can only read the value, if the format is not correct, will throw Java.util.InputMismatchException exception

Next (): When you meet the first valid character (not a space, not a newline character), start the scan, and when you meet the first delimiter or terminator (a space or newline character), end the scan to get the content scanned

Nextline (): Can scan a line of content and be captured as a string

Understanding of Hasnext (), Hasnextline (), hasnextxxx ()

is to determine if the input line still exists in the meaning of xxx

Methods related to Delimiter ()

Should be the delimiter setting for the input content,

Second, NIO (synchronous, non-blocking)

NIO is synchronous because the kernel I/O operation of its Accept/read/write method blocks the current thread

First, let's take a look at the three main components of NIO: Channel, buffer, Selector (selector)

(1) channel (channels)

Channel: The channel is an object through which data can be read and written. It can be seen as a stream in Io, unlike:

• The channel is bidirectional, readable and writable, and the stream is unidirectional.
• Channel can be read and written asynchronously
• The channel must be read and written through the buffer object

As mentioned above, all data is processed through the buffer object, so you never write the bytes directly into the channel, instead you write the data to buffer, and you do not read the bytes from the channel. Instead, the data is read from the channel into buffer, and then the byte is fetched from buffer.

Because the channel is bidirectional, the channel can better reflect the real situation of the underlying operating system than the stream. Especially in UNIX models, the underlying operating system is usually bidirectional.

There are several types of channel in Java NIO:

• FileChannel: Reading data from a file
• Datagramchannel: Read and write UDP network protocol data
• Socketchannel: Read and write TCP network protocol data
• Serversocketchannel: Can listen for TCP connections

(2) Buffer

Buffer is an object that contains some of the objects that you want to write or read to the stream. The application cannot read and write directly to the channel, but must be done through buffer, where the channel reads and writes data through buffer.

In NiO, all data is handled in buffer, which is a transit pool of NiO read and write data. Buffer is essentially an array, usually a byte data, but it can also be an array of other types. But a buffer is more than just an array, it is important that it provides structured access to the data and can also track the read and write processes of the system.

Reading and writing data using Buffer typically follows these four steps:

1. Write data to Buffer;

2. Call the flip () method;

3. Read data from Buffer;

4. Call the Clear () method or the compact () method.

When data is written to buffer, buffer records how much data is written. Once you are reading the data, you need to switch the Buffer from write mode to read mode via the Flip () method. In read mode, all data written to Buffer can be read.

Once all the data has been read, the buffer needs to be emptied so that it can be written again. There are two ways to clear a buffer: Call the Clear () or compact () method. The clear () method empties the entire buffer. The compact () method clears only the data that has been read. Any unread data is moved to the beginning of the buffer, and the newly written data is placed behind the buffer's unread data.

There are several main types of buffer:

• Bytebuffer
• Charbuffer
• DoubleBuffer
• Floatbuffer
• Intbuffer
• Longbuffer
• Shortbuffer

CopyFile instance (NIO)

CopyFile is a very good example of reading and writing, we will be through copyfile this strength to let everyone understand the operation of NIO process. CopyFile performs three basic operations: creates a buffer, then reads the data from the source file into the buffer, and then writes the buffer to the destination file.

public static void copyFileUseNIO(String src,String dst) throws IOException{//声明源文件和目标文件        FileInputStream fi=new FileInputStream(new File(src));        FileOutputStream fo=new FileOutputStream(new File(dst));        //获得传输通道channel        FileChannel inChannel=fi.getChannel();        FileChannel outChannel=fo.getChannel();        //获得容器buffer        ByteBuffer buffer=ByteBuffer.allocate(1024);        while(true){            //判断是否读完文件            int eof =inChannel.read(buffer);            if(eof==-1){                break;              }            //重设一下buffer的position=0，limit=position            buffer.flip();            //开始写            outChannel.write(buffer);            //写完要重置buffer，重设position=0,limit=capacity            buffer.clear();        }        inChannel.close();        outChannel.close();        fi.close();        fo.close();}   

(c) Selector (Selector object)

The first thing you need to know is that the thread context switching overhead becomes apparent at high concurrency, which is a low extensibility disadvantage of synchronous blocking.

Selector is an object that can register to many channel, listen to the events that occur on each channel, and can determine the channel read and write according to the event situation. In this way, multiple channel management through one thread can handle a large number of network connections.

Selector advantages

With selector, we can use a thread to handle all the channels. Switching between threads is expensive for the operating system, and each thread consumes a certain amount of system resources. Therefore, the fewer threads you use for the system, the better.

1. How to create a selector

Selector is where you register interest in various I/O events, and when those events occur, it is the object that tells you what happened.

Selector selector = Selector.open();

2. Register Channel to Selector

In order for the channel and selector to be used, we need to register the channel with the selector. To implement registration by calling the Channel.register () method:

channel.configureBlocking(false);SelectionKey key =channel.register(selector,SelectionKey.OP_READ);

Note that the registered channel must be set to asynchronous mode, otherwise asynchronous IO will not work, which means we cannot register a filechannel to selector because FileChannel does not have an asynchronous mode, But the Socketchannel in network programming is possible.

3. About Selectionkey

Note that the return value of the call to register () is a selectionkey. Selectionkey represents this channel registration on this Selector. When a Selector notifies you of an incoming event, it is done by providing the selectionkey corresponding to the event. Selectionkey can also be used to cancel the registration of a channel.

The following properties are included in the Selectionkey:

• The interest set
• The Ready Set
• The Channel
• The Selector
• An attached object (optional)

(1) Interest set

As we mentioned earlier to register the channel with selector to listen to events of interest, interest set is the collection of events of interest that you want to select. You can read and write interest set by Selectionkey object:

int interestSet = selectionKey.interestOps();boolean isInterestedInAccept  = interestSet & SelectionKey.OP_ACCEPT;boolean isInterestedInConnect = interestSet & SelectionKey.OP_CONNECT;boolean isInterestedInRead    = interestSet & SelectionKey.OP_READ;boolean isInterestedInWrite   

As we can see from the above example, we can find the events we are interested in from Selectionkey by using constants in and and Selectionkey.

(2) Ready Set

The ready set is a collection of operations that the channel is already prepared for. After you select selection, you should first access this ready set. The selection will be explained in the next section. You can access the ready collection in this way:

int readySet = selectionKey.readyOps();

You can use methods like the interest collection to detect what events or operations in the channel are ready. However, you can also use the following four methods, all of which return a Boolean type:

selectionKey.isAcceptable();selectionKey.isConnectable();selectionKey.isReadable();selectionKey.isWritable();

(3) Channel and Selector

We can obtain selector and registered channel via Selectionkey:

Channel  channel  

(4) Attach an object

You can attach an object or more information to the Selectionkey, which makes it easy to identify a given channel. For example, you can attach a buffer that is used with a channel, or an object that contains aggregated data. Here's how to use it:

selectionKey.attach(theObject);Object attachedObj = selectionKey.attachment();

You can also attach objects when registering a channel with the selector using the register () method. Such as:

SelectionKey key = channel.register(selector, SelectionKey.OP_READ, theObject);

4. About Selectedkeys ()

Additional readiness checks are generally performed in the production system

Once the Select () method is called, it returns a numeric value indicating that one or more channels are ready, and then you can get the ready channel by calling the Selectionkey collection returned by the Selector.selectedkeys () method. Please see the demo method:

Set<SelectionKey> selectedKeys = selector.selectedKeys();

When you register a channel through selector, the Channel.register () method returns a Selectionkey object that represents the channel you register. These objects can be obtained through the Selectedkeys () method. You can iterate over these selected keys to get a ready channel, here's the demo code:

Set<SelectionKey> selectedKeys = selector.selectedKeys();Iterator<SelectionKey> keyIterator = selectedKeys.iterator();while(keyIterator.hasNext()) { SelectionKey key = keyIterator.next();if(key.isAcceptable()) {// a connection was accepted by a ServerSocketChannel.} else if (key.isConnectable()) {// a connection was established with a remote server.} else if (key.isReadable()) {// a channel is ready for reading} else if (key.isWritable()) {// a channel is ready for writing}keyIterator.remove();}

This loop iterates through each key in the set of selected key and tests each key to determine which channel is ready.

Notice the last Keyiterator.remove () method in the loop. The Selector object does not automatically remove the Selectionkey instance from its own selected key collection. We need to remove it ourselves when we have finished processing a channel. The next time the channel is ready, selector will add it to the selected key collection again.

The channel returned by the Selectionkey.channel () method needs to be converted to the type you are specifically dealing with, such as Serversocketchannel or Socketchannel, and so on.

(4) NiO multiplexing

Main steps and elements:

• First, create a Selector with Selector.open () as a dispatcher-like role.

• Then, create a serversocketchannel, and register with Selector, and by specifying Selectionkey.op_accept, tell the dispatcher that it is concerned with the new connection request.

• Note, why should we explicitly configure non-blocking mode? This is because the registration operation is not allowed in blocking mode and throws a Illegalblockingmodeexception exception.

• The Selector block is in the select operation and is awakened when there is a Channel access request.

• In a specific way, data operations are performed via Socketchannel and Buffer

IO is synchronous blocking mode, so multiple threads are required for multitasking. NIO is the use of single-threaded polling event mechanism, through the efficient positioning of the Channel, to determine what to do, only the select stage is blocked, can effectively avoid a large number of client connections, frequent thread switching problems, the application of the expansion capacity has been greatly improved

Three, NIO2 (asynchronous, non-blocking)

AIO is an abbreviation for asynchronous IO, although NIO provides a non-blocking method in network operations, but NiO's IO behavior is synchronous. For NIO, our business thread is notified when the IO operation is ready, then the IO operation is done by the thread itself, and the IO operation itself is synchronous.

However, for AIO, it is a step further, it is not when the IO is ready to notify the thread, but after the IO operation has been completed, and then to notify the thread. Therefore, AIO is not blocked, and our business logic will become a callback function, waiting for the completion of the IO operation by the system automatically triggered.

Unlike NiO, when it comes to read and write operations, only the read or write method of the API can be called directly. Both methods are asynchronous, and for a read operation, when a stream is readable, the operating system streams the readable buffer into the Read method and notifies the application that, for a write operation, the operating system proactively notifies the application when it finishes writing the stream passed by the write method. It can be understood that the Read/write method is asynchronous and will invoke the callback function after completion. In JDK1.7, this part of the content is called nio.2, with the following four asynchronous channels added mainly under the Java.nio.channels package:

• Asynchronoussocketchannel
• Asynchronousserversocketchannel
• Asynchronousfilechannel
• Asynchronousdatagramchannel

In AIO socket programming, the server-side channel is Asynchronousserversocketchannel, which provides an open () static factory, a bind () method for binding the server-side IP address (and the port number), The Accept () is also available to receive user connection requests. The channel used by the client is Asynchronoussocketchannel, and this channel processing provides the open static Factory method, which also provides the read and write methods.

In AIO programming, after issuing an event (accept read Write, etc.) to specify the event handler class (callback function), the event handler class in Aio is Completionhandler<v,a>, this interface defines the following two methods, Callbacks are called when the asynchronous operation succeeds and fails.

void completed (V result, A attachment);

void failed (Throwable exc, A attachment);

Java IO, NIO, Aio detailed