Java Advanced Knowledge Point 5: The cornerstone of high concurrency on the server side-NIO and reactor modes and AIO and Proactor modes

I. BACKGROUND

In order to improve the concurrency of the server, there are usually two main directions of thinking.

1, System architecture level. such as load balancing, multilevel caching, cell deployment, and so on.

2, single-node optimization level. such as fixed code-level performance bugs, JVM parameter tuning, IO optimization, and so on.

In general, the reasonable degree of system architecture, determines the overall performance of the system scalability (high scalability, in short, can be very willful, performance is not added to the machine, add to the performance of enough), and the performance of a single node optimization, determine the single request delay, and to achieve the desired performance, The size of the desired cluster size. A two-pronged approach can quickly build a well-performing system.

Today, let's talk about the most important IO optimizations at the single-node optimization level. IO optimization is most important because the IO speed is much lower than CPU and memory, and not good software design, often lead to CPU and memory by IO drag, how to get rid of IO constraints, the full use of CPU and memory potential, is the core content of performance optimization.

And how is CPU and memory dragged down by IO? This starts with several typical IO operating modes in Java.

Ii. Typical IO operation mode in Java 2.1 synchronous blocking mode

The bio-style API in Java is the pattern, for example:

Socket socket =// cannot read data vow not to return

In this mode, the most intuitive feeling is that if the IO device temporarily has no data to read, the call API will be stuck, if the data has not come to stay stuck.

2.2 Synchronous non-blocking mode

The NIO-style API in Java is the pattern, for example:

// get channel for non-blocking status //cannot read the data even if, immediately return 0 tell you not to read

In this mode, it is often necessary to constantly call the API until the data is read, but fortunately the function call does not get stuck, I want to continue to try to read or to do something else to read it.

2.3 Asynchronous non-blocking mode

The AIO-style API in Java is the pattern, for example:

Asynchronoussocketchannel Asynchronoussocketchannel = getasynchronoussocketchannel (); Asynchronoussocketchannel.read (bytebuffer.allocate (nullnew Completionhandler<integer, Object>() {    @Override    publicvoid  completed (Integer result, Object Attachment) {        ///cannot read the data does not trigger the callback to annoy you, only if you do read the data, and the data already exists Bytebuffer, the API will be actively notify you via this callback interface     }    @Override    publicvoid  failed (Throwable exc, Object Attachment) {    }});

This mode of service is most in place, in addition to making programming relatively complex, almost impeccable.

2.4 Summary

For IO operations, the essential difference between synchronous and asynchronous is whether the API will proactively notify you of the state of IO readiness (such as data readable). Synchronization means to know if the IO is ready, you have to initiate a query, a typical question and answer, and if the response is not ready, you have to keep asking until the answer is ready. Async means that after IO is ready, the API will proactively notify you without you ever having to initiate an inquiry, which usually requires the callback interface to pass in the notification when the API is called.

The essential difference between blocking and non-blocking is that the API will suspend the current thread if the IO operation cannot be completed immediately because the IO is not ready. Blocking means that the API will wait until IO is ready to complete the IO operation before returning, and the user thread that called the API will be suspended until other calculations are processed. Non-blocking means that the API returns immediately, rather than waiting for IO to be ready, and the user can immediately gain control of the thread again, and can use that thread for additional computation processing.

Does it have an asynchronous blocking mode? If the API supports Asynchrony, the API says, "You play, I'm ready to let you know," but you're still dumb to play, waiting for the API to be done. This is usually because the IO operation is very important, can not get the results of the business process is not able to continue, so for the sake of simplicity of programming, still wait for it. Visible asynchronous blocking patterns are more about business process control and simplifying coding difficulties, which are formed by business code autonomy, and the Java language does not specifically prepare you for asynchronous blocking of Io APIs.

Three, separation fast and slow 3.1 bio limitations

CPU and memory are high-speed devices, disk, network and other IO devices are low-speed devices, in the Java programming language, the use of CPU and memory is abstracted to the use of threads, stacks, heaps, the use of IO devices is abstracted as IO-related API calls.

Obviously, if the bio-style IO API is used, because of its synchronous blocking feature, when the IO device is not ready, the thread hangs and the thread cannot continue to use the CPU and memory until IO is ready. Because the IO device is much slower than CPU and memory, when using the bio-style API, there is a huge probability that the current thread will hang for a long time, resulting in a situation where CPU and memory resources are being dragged down by IO.

As a service-side application, there will be a large number of clients to the server to initiate the connection request scenario, each connection to the service side, it means that the subsequent network IO read, after the completion of the IO read, to obtain the complete request content, in order to perform some column-related calculation processing to obtain the request results, Finally, the results are written back to the client via network IO. Using Bio's coding style, usually the same thread is responsible for a connected IO read, data processing, and IO writeback, which most of the time may be waiting for IO to be ready, with very little time to really take advantage of CPU resources.

When the server wants to handle a large number of client connections at the same time, the backend opens a thread corresponding to the number of concurrent connections. Threads are a valuable resource for the operating system, and Java consumes out-of-heap memory-XSS The specified thread stack size for each operating system thread, and if there are a large number of threads at the same time, the overhead of the operating system scheduler thread increases significantly, resulting in a rapid degradation of server performance. So the server wants to support tens of thousands or even hundreds of thousands of of high concurrent connections, it is difficult.

Breakthrough 3.2.1 of 3.2 NiO breakthrough idea

Because of the non-blocking nature of NIO, when the IO is not ready, the thread can continue to handle other things without having to hang. This provides the possibility that the separation is fast and slow, that the high-speed CPU and memory do not have to suffer the same IO interaction, and that a thread does not have to be limited to a single IO connection service. This makes it possible to handle massive IO connections with a small number of threads.

3.2.2 Idea landed

Although we have seen the dawn, we need to solve some practical problems to get this idea landed.

A) When IO is not ready, the thread is freed and switched to other connections, who is going to monitor this abandoned IO readiness event?

b) Io is ready, who is going to be responsible for assigning this IO to the appropriate thread to continue processing?

To solve the first problem, the operating system provides IO multiplexers (for example, select, poll, and Epoll under Linux), which are encapsulated by Java (typically using the best performing epoll) and the corresponding IO multiplexing API. The typical programming pattern for the Multiplexed API for NIO is as follows:

Selector Selector = Selector.open ();//creating a multiplexerSocketchannel channel = Getchannel ();//get Socketchannel in NiO, representing an IO connection//register the channel with the Multiplexer and delegate the multiplexer to monitor the read-ready and write-ready events for the connection//of course a multiplexer can monitor multiple channel, and monitored events can also be customized by the second OPS parameter//when the registration is complete, a selectionkey is returned, which is bound to the channel that the delegated multiplexer monitorsSelectionkey Selectionkey = Channel.register (selector, Selectionkey.op_read |selectionkey.op_write);//the multiplexer starts to monitor, with IO events ready to return, continuous monitoring through continuous cycle completion while(Selector.select ()! = 0) {    //Traverse all the Ready channel-associated SelectionkeyIterator<selectionkey> Iterator =Selector.selectedkeys (). iterator ();  while(Iterator.hasnext ()) {Selectionkey key=Iterator.next (); //If this channel is read ready        if(Key.isreadable ()) {//Read the channel((Socketchannel) Key.channel ()). Read (Bytebuffer.allocate (10)); }        if(Key.iswritable ()) {//... ...        }        if(Key.isacceptable ()) {//... ...        }//Remove Selectionkey that have already been processedIterator.remove (); }}

The IO Multiplexing API can be implemented with one thread to monitor IO readiness events for all IO connections.

The second problem in the code above is actually "resolved", but the above code is the use of monitoring IO-ready event thread to complete the IO operation, if the IO operation time is large (such as the read operation is ready, there is a lot of data to read), then will cause the monitoring thread for a specific IO service for a long time, This causes the entire system to be unable to perceive other IO readiness events for a long time and to dispatch IO processing tasks. So in a production environment, a boss thread is typically used to monitor IO-ready events, and a work thread pool is responsible for specific IO read-write processing. After the boss thread detects a new IO-ready event, it completes the assignment of the IO Operation task based on the event type and assigns the specific action to the work thread. This is actually the core idea of the reactor model.

3.2.3 Reactor Mode

As mentioned above, the core concept of the reactor model is:

A) dependent on non-blocking IO.

b) Use the multiplexer to monitor the mass IO readiness event.

c) Use the Boss thread and the work thread pool to separate the monitoring of IO events and the processing of IO events.

There are three types of roles in the reactor mode:

a) acceptor. The user processes the client connection request. The acceptor role is mapped to Java code, which is Socketserverchannel.

b) Reactor. The processing task used to dispatch an IO-ready event. The reactor role is mapped to Java code, which is the boss thread that uses the multiplexer.

c) Handler. Used to handle specific IO-ready events. (such as reading and processing data, etc.). The handler role is mapped to Java code, which is each thread in the worker thread pool.

Acceptor connection readiness Events, which are also referred to reactor supervision, some places in order to separate the establishment of the connection and the processing of the connection, for the separation of reactor into a master reactor, the specialized user supervises the connection related events (ie selectionkey.op_ ACCEPT), a data-related event (that is, Selectionkey.op_read and Selectionkey.op_write) from reactor, a dedicated user-supervised connection.

About reactor model diagram, online a search a lot of, I will not caught dead. Believe in understanding its core ideas, the picture naturally in the heart. With regard to the application of the reactor model, you can refer to the NIO programming framework Netty, in fact, after Netty, the Netty framework is generally used directly for the service-side NIO programming.

3.3 Aio's further 3.3.1 Aio's unique advantage

It's easy to see that if you're using a Aio,nio breakthrough, the landing problem seems natural. Because each IO operation can register a callback function, naturally there is no need to specifically have a multiplexer to listen to the IO-ready event, and do not need a boss thread to allocate events, all IO operations as soon as the completion, will naturally through the callback into their own next processing.

And, more surprisingly, through AIO, even the work thread in NiO to read and write data can be omitted, because AIO is to ensure that the real data read/write after the callback function is triggered, the user does not have to focus on the IO operation itself, only need to focus on the data in the IO, the business logic should be carried out.

In short, NiO's multiplexer is to notify you of an IO ready event, and the AIO callback is to notify you of an IO completion event. AIO is done more thoroughly. This also leads to performance improvements on some platforms because the IO read and write operations of AIO can be done by the operating system kernel, giving full play to the kernel potential and reducing the context transitions between the user and kernel states when the IO system is called, which is more efficient.

(Unfortunately, there are a lot of problems with the AIO implementation of the Linux kernel (not covered in this article), performance is not as good as nio in some scenarios, and even Java on Linux uses Epoll to emulate AIO, so using the Java AIO API on Linux, It is only possible to experience the programming style of asynchronous IO, but it is not more efficient than NIO. In summary, Java Server programming on Linux platform, the current mainstream still use NIO model. ）

The typical programming mode for using the AIO API is as follows:

//Create a group, similar to a thread pool, for processing IO completion EventsAsynchronouschannelgroup Group = Asynchronouschannelgroup.withcachedthreadpool (Executors.newcachedthreadpool (), 32);//turn on a asynchronousserversocketchannel and listen on port 8080Asynchronousserversocketchannel Server =Asynchronousserversocketchannel.open (group); Server.bind (NewInetsocketaddress ("0.0.0.0", 8080));//receive a new connectionServer.accept (NULL,NewCompletionhandler<asynchronoussocketchannel, object>() {    //handler function for new connection ready event@Override Public voidcompleted (asynchronoussocketchannel result, Object attachment) {Result.read (Bytebuffer.allocate (4), Attachment,NewCompletionhandler<integer, object>() {            //Read handler function for completion event@Override Public voidcompleted (Integer result, Object attachment) {} @Override Public voidfailed (Throwable exc, Object attachment) {}}); } @Override Public voidfailed (Throwable exc, Object attachment) {}});

3.3.2 Proactor Mode

The Java AIO API is actually the application of Proactor mode.

Similar to the reactor pattern, the proactor pattern can also abstract three types of characters:

a) acceptor. The user processes the client connection request. The acceptor role is mapped to Java code, which is Asynchronousserversocketchannel.

b) Proactor. The processing task used to dispatch an IO completion event. The Proactor role maps to Java code, which is the addition of callback parameters to the API methods.

c) Handler. Used to process a specific IO completion event. (such as processing of read data, etc.). The handler role is mapped to Java code, which is each thread in the Asynchronouschannelgroup.

As can be seen, the biggest difference between Proactor and reactor is:

A) There is no need to use a multiplexer.

b) handler does not need to perform specific IO operations (such as reading data or writing data), but only performing business processing of IO data.

Iv. Summary

1, the IO in Java has synchronous blocking, synchronous non-blocking, asynchronous non-blocking three operation modes, respectively, corresponding to bio, NIO, AIO three types of API style.

2, bio needs to ensure a connection to a thread, because the thread is a valuable resource of the operating system, not open too much, so bio severely restricts the number of concurrent connections that can be hosted on the server.

3, the use of NIO characteristics, supplemented by reactor programming mode, Java in Linux to achieve high concurrency server-side capacity of the mainstream way.

4, the use of AIO features, supplemented by Proactor programming mode, on other platforms (such as Windows) to achieve higher performance than NIO.

Java Advanced Knowledge Point 5: The cornerstone of high concurrency on the server-NIO and reactor modes and AIO and Proactor modes