Netty IO Threading Model Learning Summary

The main thread of the Netty framework is the IO thread, and the threading model directly determines the throughput, concurrency, and security of the system.

The threading model of Netty follows the underlying threading model of reactor. Now let's look at the model together.

Reactor threading Model reactor single-threaded model

All IO operations in a single-threaded model operate on a NIO thread:

Contains requests to accept clients, read messages and replies from clients. Because asynchronous non-blocking IO is used, all IO operations are not blocked and in theory a single thread can handle all IO operations.

Single-threaded models for small-capacity applications. Because in high concurrency applications can cause the following problems

1. One thread handles hundreds or thousands of links at the same time, which is not supported on performance. Even IO thread CPU 100% does not meet the requirements.

2. When the NIO line layer is overloaded, processing speed slows down, causing a large number of clients to time out, re-send, which can aggravate the load of nio, resulting in a large system timeout

3. Once the IO thread runs, it will cause the entire system communication module to be unavailable, resulting in node failure

Reactor multithreaded Model

The model organizes a set of threads for IO operations
Characteristics:
1. There is a dedicated NIO thread---acceptor thread for listening to the server, accepting TCP requests from the client
2. The read and write of the network operation is responsible for the read and receive encoding and sending of the message by an IO thread pool.
3. An IO thread can handle n links at the same time, but one link only corresponds to one IO thread. Prevent concurrency problems with operations

Suitable for most scenarios, but for the concurrent million or the server needs to secure the client handshake authentication, the authentication is very performance-consuming situation, will lead to performance bottlenecks!

Multithreading model of primary and secondary reactor

The connection that accepts the client is not a separate IO thread, but a NIO thread pool:

After the Acceptor accepts the client's request and processing is complete, the newly created Socketchannel is registered to a thread on the IO thread pool by the

He is responsible for the IO read-write connection coding work. The acceptor thread pool is only responsible for the client's login handshake and secure authentication once the link is

The link to the thread pool on the backend, and has him perform subsequent IO operations.

Netty Threading Model


public void bind (int port) throws Exception {
Configure the NIO thread group on the service side
Eventloopgroup Bossgroup = new Nioeventloopgroup ();
Eventloopgroup Workergroup = new Nioeventloopgroup ();
try {
Serverbootstrap B = new Serverbootstrap ();
B.group (Bossgroup, Workergroup)
. Channel (Nioserversocketchannel.class)
. option (Channeloption.so_backlog, 1024)
. Childhandler (New Childchannelhandler ());
Bind port, synchronization waits for success
Channelfuture f = b.bind (port). sync ();
Wait for the server to listen port shutdown
F.channel (). Closefuture (). sync ();
} finally {
Graceful exit, releasing thread pool resources
Bossgroup.shutdowngracefully ();
Workergroup.shutdowngracefully ();
}
}
Private class Childchannelhandler extends Channelinitializer<socketchannel> {
@Override
protected void Initchannel (Socketchannel arg0) throws Exception {
Arg0.pipeline (). AddLast (New Timeserverhandler ());
}
}


The Netty server was started by creating two Nioeventloopgroup independent reator thread pools, a TCP connection for receiving clients, and a related read and write operation for processing IO.

Netty threading model is based on the reactor model, the threading model is not static, through the configuration of the start parameter, you can switch in three kinds.

The boot process, Bossgroup will select a eventloop need to bind Serversocketchannel to receive the client connection, after processing, the prepared Socketchnanell smoothly registered to workgroup.

Netty the creation process of the service side

Timing Diagram:

Netty the underlying details of the shielded NIO communication:

2. First, create the Serverbootstrap, which is the startup helper class for the Netty server.

4. Sets and binds the reactor thread pool. Netty's reactor thread pool is eventloopgroup, which is actually an array of eventloop threads. EventLoop's responsibility is to handle all channel registrations on the selector of this thread multiplexer

6. Set the Nioserversocketchannel. Netty creating Nioserversocketchannel objects with reflection through the factory class

8. Setting TCP Parameters

10. Create and initialize the Channelpipeline when the link is established. It is essentially a chain of responsibilities that handles network events and is responsible for managing and executing Channelhandler. Network events flow in Channelpipeline in the form of event streams, and the execution of Channelhandler is scheduled by channelpipeline according to Channelhandler execution strategy.

    1. Bind and start the listening port
    2. Bind the port and start. The nioeventloop is responsible for scheduling and performing the selector polling operation and selecting the Ready Channel collection. After polling for the Ready channel, the reactor thread Nioeventloop executes Channelpipeline's corresponding method, eventually dispatching and executing channelhandler.

Nioeventloop IO Threading Analysis

As a Netty reactor thread, because to handle network IO read and write, a Multiplexer object is aggregated, which obtains a multiplexer through open. His operations are performed primarily in the For loop of the Run method.

1. As a bossgroup thread, he needs to bind Nioserversocketchannel to listen for connet requests from clients and handle connections and checksums.
2. As a thread of the workgroup line layer, the connection-ready socketchannel needs to be bound to the thread, so a client connects to one thread, and one thread can bind multiple client connections.

From the scheduling level, there is no other type of thread in the EventLoop thread to perform other tasks asynchronously, which avoids multi-threaded concurrency and lock contention, and improves the processing and scheduling performance of I/O threads.

Nioeventloop Thread Protection

The IO operation is the core of the thread, and in the event of a failure, the multiplexer and multiple links above it do not work properly, so he needs special protection. He has done protection in the following two areas:

1. Handling Exceptions with caution

An exception can cause a thread to fly, causing all links under the thread to be unavailable, and Try{}catch (Throwable) catches the exception and prevents the flight from running. After an exception occurs, you can resume execution. The Netty principle is that an exception to a message does not cause the entire link to be unavailable, that one link is not available, and that no other links are unavailable.

1. Circumventing NiO bugs

Selector.select a epoll BUG that might trigger the JDK when there is no task to execute. This is the famous JDK Epoll bug,jdk1.7 earlier version claimed to have been solved, but according to the online feedback, there is this BUG. The server directly manifests as the CPU of the IO thread is very high, may reach 100%, may cause the node to fail!!!

Why Epoll bugs occur

The Netty fix strategy is:

1. Statistics on the operating cycle of the selector select

2. Count once for each empty select operation

3. In a certain period (such as 100ms) continuous n This empty polling, indicating that triggered the Epoll dead loop bug

4. After the dead loop is detected, the way to rebuild the selector is to restore the system to normal

Netty This strategy, this bug is perfectly avoided.

Reference: Netty authoritative Guide 2

Netty IO Threading Model Learning Summary