The way to read Netty's high-performance architecture

Netty is a high-performance, asynchronous event-driven NIO framework that provides support for TCP, UDP, and file transfers, and as an asynchronous NIO framework, all IO operations of Netty are asynchronous and non-blocking, through the future-listener mechanism, The user can obtain the IO operation result conveniently or through the notification mechanism.

As the most popular NIO framework, Netty has been widely used in the field of Internet, big Data distributed computing, game industry and communication industry, and some industry-famous open source components are also built on the Netty NIO framework.

Why Choose Netty

Netty is one of the industry's most popular NIO frameworks, and its robustness, functionality, performance, customisation, and scalability are among the best in its kind, and it has been validated by hundreds of commercial projects, such as the RPC framework for Hadoop Avro using Netty as the underlying communication framework , many other industry-leading RPC frameworks also use Netty to build high-performance asynchronous communication capabilities.

Through the analysis of Netty, we summarize its advantages as follows:

• Easy to use API, low development threshold;

• Powerful, preset a variety of codec functions, support a variety of mainstream protocols;

• The ability of customization is strong, and the communication framework can be flexibly expanded by Channelhandler;

• High performance, compared with other industry mainstream NIO framework, the Netty is optimal.

• Mature, stable, netty fixes all the JDK NiO bugs that have been discovered, and business developers don't need to worry about nio bugs anymore;

• The community is active, the version iteration cycle is short, the bugs found can be repaired in time, and more new features will be added;

• Experienced a large-scale commercial application test, quality is verified. In the Internet, big data, online games, enterprise applications, telecommunications software and many other industries have been successful commercial, proved that it has been fully able to meet the commercial applications of different industries.

Netty Architecture Analysis

The Netty is designed using a more typical three-layer network architecture, as shown in the logical architecture diagram:

First layer: Reactor communication scheduling layer, which is done by a series of auxiliary classes, including Reactor thread Nioeventloop and its parent class, Niosocketchannel/nioserversocketchannel, and its parent class, Bytebuffer and the various types of Buffer, Unsafe, and various internal classes derived therefrom. The main function of this layer is to listen to the network read and write and connect operations, responsible for the network layer of data read into the memory buffer, and then trigger a variety of network events, such as connection creation, connection activation, read events, write events, and so on, to trigger these events into the PipeLine, by PipeLine as a responsibility chain to for subsequent processing.

The second layer: the responsibility chain PipeLine, it is responsible for the event in the chain of responsibility of the orderly dissemination, while responsible for the dynamic orchestration of responsibility chain, the chain of responsibility can choose to listen and handle the events of their own concern, it can intercept processing and back/forward propagation events, different applications of the Handler node function is also different, usually , will often develop codec Hanlder for message codec, it can convert external protocol messages into internal POJO objects, so that the upper business side only need to care about the business logic, do not need to perceive the underlying protocol differences and threading model differences, to achieve the layered isolation of the architecture level.

The third layer: business logic processing layer, can be divided into two categories:

1. Pure business logic processing, such as order Processing.

2. Application layer protocol management, such as HTTP protocol, FTP protocol, etc.

Next, I talk about the architecture of Netty from three aspects that affect communication performance (I/O model, thread scheduling model, serialization mode).

I/O model

The traditional synchronous blocking I/O mode looks like this:

There are many drawbacks to it:

• Performance problem: A connection-first-thread model leads to a significant limit on the number of concurrent accesses and system throughput of the server;

• Reliability problem: Due to the synchronous blocking mode of I/O operation, when the network congestion or slow communication processing causes the I/O thread to be suspended, the blocking time is unpredictable;

• Maintainability problem: The number of I/O threads cannot be effectively controlled, resources cannot be effectively shared (multithreading concurrency problem), System maintainability is poor;

Comparison of functions and characteristics of several I/O models:

The Netty I/O model is based on non-blocking I/O implementations, with the underlying dependency on the selector of the JDK NIO framework.

Selector provides the ability to select tasks that are already ready. simply put, selector will continually poll the channel registered on it , and if a channel has a new TCP connection access, read and write events, the channel is in a ready state and will be polled by selector. The Selectionkey can then be used to obtain a set of ready channel for subsequent I/O operations.

A multiplexer selector can poll multiple channel at the same time, because the Jdk1.5_update10 version (+) uses Epoll () instead of the traditional select implementation, it does not have the maximum connection handle 1024/2048 limit. This means that only one thread is responsible for selector polling, which can be accessed by tens of thousands of clients, which is really a huge technological advance.

With the non-blocking I/O model, Netty solves the performance, throughput, and reliability issues associated with traditional synchronous blocking I/O.

Thread Scheduling model

There are three commonly used reactor threading models, such as the following:

• Reactor single-threaded model: Reactor single-threaded model, which means that all I/O operations are done on the same NIO thread. For some small-capacity scenarios, you can use a single-threaded model.

• Reactor Multithreading Model: The biggest difference between a rector multithreaded model and a single-threaded model is that there is a set of NIO threads that handle I/O operations. It is mainly used for high concurrency and large traffic scenarios.

• Master-Slave reactor multithreading model: The main feature of the master-slave reactor threading model is that the server is no longer a 1 separate NIO thread for receiving client connections, but rather a separate NIO thread pool. With the master-slave NIO threading model, 1 server-side listener threads are unable to effectively handle all client connection performance problems.

In fact, the threading model of Netty is not fixed, and it is possible to support the above three reactor threading models by creating different Eventloopgroup instances in the boot helper classes and configuring them with the appropriate parameters.

In most scenarios, parallel multithreading can improve the concurrency performance of the system. However, if concurrent access to shared resources is handled improperly, it can lead to severe lock contention, which can eventually result in degraded performance. In order to avoid the performance loss of lock competition as much as possible, it can be accomplished by serialization design, that is, the processing of message is done in the same thread as possible, and the thread switching is not done, so the multi-threading competition and the synchronous lock are avoided.

To maximize performance, the Netty adopts a serial, non-locking design that performs serial operations within the I/O thread to avoid performance degradation caused by multi-threaded contention. On the surface, the serialization design seems to be low CPU utilization and not enough concurrency. However, by adjusting the thread parameters of the NIO thread pool, you can simultaneously start multiple serialized threads running concurrently, a locally unlocked serial threading design that performs better than one queue-multiple worker threading models.

Serialization mode

The key factors that affect serialization performance are summarized below:

• serialized bitstream size (network bandwidth consumption)

• Serialization & Deserialization Performance (CPU resource consumption)

• Performance of concurrent calls: stability, linear growth, even Shiyanmao spikes

For the Java serialization and binary coding performance test, coded 1 million times, the test results show that: The performance of Java serialization is only about 6.17% of the binary encoding.

Netty provides support for Google Protobuf by default, and by extending the Netty codec interface, users can implement other high-performance serialization frameworks, such as Thrift's compressed binary codec framework.

Different scenarios have different requirements for the serialization framework, and for high-performance scenarios Netty provides Google's Protobuf binary serialization framework by default, and can be implemented based on the codec framework provided by the Netty if the user needs additional binary serialization frameworks.

Reliability of Netty Architecture analysis

The reliability challenges faced by Netty:

• As the underlying network communication framework of the RPC framework, a failure will result in the inability to make remote service (interface) calls.

• As the underlying communication framework for the application layer protocol, the application protocol stack will not work properly once the failure occurs.

• The network environment is complex (for example, the Gsm/3g/wifi network of hand-tour or push service), the fault is unavoidable and the business cannot be interrupted.

From the application scenario, Netty is the basic communication framework, and if a bug occurs, the light needs to restart the application, which can lead to a disruption of the entire business. Its reliability affects the data communication and exchange of the whole business cluster, in today's distributed software architecture, communication interruption means the whole business interruption, and the reliability of communication is very high in distributed architecture.

From the operating environment, Netty will face a poor network environment, which requires its own reliability is good enough, the platform can solve the reliability of the problem needs to be solved by the netty itself, otherwise it will cause the top users to pay attention to the underlying failure, which will reduce the ease of Netty, At the same time increase the user's development and operation and maintenance costs.

The reliability of Netty is so important that any failure of it can lead to business disruption and huge economic losses. As a result, Netty continues to incorporate new reliability features into the iteration of the version to meet the growing demand for high reliability and robustness of the user.

Link Validity detection

The heartbeat detection mechanism provided by Netty is divided into three types:

• Read idle, link duration t not read to any message;

• Write idle, link duration t does not send any messages;

• Read/write idle, link duration t not receiving or sending any messages.

When a network occurs a single pass, the connection is stuck in a firewall, a long time GC, or an unexpected exception occurs on the communication thread, the link is not available and is not readily discoverable. In particular, anomalies occur during the early morning business downturn, and when the peak business hours arrive, because the link is not available can result in an instantaneous large volume of business failures or timeouts, which will pose a significant threat to system reliability.

From the technical aspect, to solve the link reliability problem, we must periodically check the validity of the link. At present, the most popular and common practice is heartbeat detection.

The heartbeat detection mechanism is divided into three levels:

• The TCP level heartbeat detection, namely TCP's keep-alive mechanism, its scope is the entire TCP protocol stack;

• The heartbeat detection of the protocol layer mainly exists in the raised here connection protocol. such as the SMPP Agreement;

• The application layer of heartbeat detection, it is mainly by the business products through the agreed way to send each other the heartbeat message implementation.

The purpose of heartbeat detection is to confirm that the current link is available, that the other person is alive and able to receive and send messages normally. As a highly reliable NIO framework, Netty also provides a heartbeat detection mechanism based on link idle:

• Read idle, link duration t not read to any message;

• Write idle, link duration t does not send any messages;

• Read/write idle, link duration t not receiving or sending any messages.

Traffic shaping

Traffic shaping (traffic Shaping) is a measure of actively adjusting the rate of flow output. Netty's traffic shaping has two functions:

• Prevent the downstream network element is crushed due to the uneven performance of the upper and lower network elements, the business process is interrupted;

• Prevents the "brace-up" problem caused by the back-end business thread processing because the communication module is receiving too fast messages.

The principle of traffic shaping is as follows:

Traffic shaping (traffic Shaping) is a measure of actively adjusting the rate of flow output. A typical application is to control the output of local traffic based on the TP indicator of downstream network nodes. The main difference between traffic shaping and traffic policing is that traffic shaping caches the messages that need to be discarded in traffic policing-usually by putting them in buffers or queues, also known as traffic shaping (traffic Shaping, or TS, for short). When the token bucket has enough tokens, these cached messages are sent out evenly. Another difference between traffic shaping and traffic policing is that shaping can increase latency, while regulation rarely introduces additional delays.

Netty supports two types of traffic shaping modes:

• Global traffic Shaping: the scope of global traffic shaping is process-level, no matter how many channel you create, its scope is for all channel. Parameters can be set by the user: the receiving rate of the message, the sending rate of the message, and the shaping period.

• Link-level traffic shaping: The biggest difference between single-link traffic shaping and global traffic shaping is that it is scoped to a single link and can have different shaping strategies for different links.

Graceful shutdown

Netty's Elegant Stop trilogy:

• No longer receiving new messages

• Pre-processing operations before exiting

• Freeing operations for resources

The graceful outage of Java is usually realized by registering the shutdownhook of the JDK, when the system receives the exit instruction, it first marks the system in the exit state, no longer receives the new message, then finishes processing the backlog of messages, finally calls the resource recycling interface to destroy the resource, and the last thread exits execution.

Usually graceful exit requires a timeout control mechanism, such as 30S, if the arrival time-out is still not completed before the exit of the resource collection, and other operations, the shutdown script directly calls the Kill-9 PID, forced exit.

In real projects, Netty as a high-performance asynchronous NIO communication framework, often used as the basic communication framework for the access, resolution and scheduling of various protocols, such as in RPC and distributed service frameworks, often using Netty as the underlying communication framework for internal private protocols.

When the application process gracefully exits, the Netty as a communication framework also needs to be gracefully exited, mainly for the following reasons:

• Release NiO threads, handles and other resources as soon as possible;

• If you use flush to do bulk message sending, you need to send the backlog of messages that have been accumulated in the send queue to complete;

• The message that is being write or read needs to be processed;

• Set the scheduled task in the Nioeventloop thread scheduler, which needs to be performed or cleaned up.

Security of Netty Architecture analysis

Security challenges faced by Netty:

• Open to third parties

• Basic communication Framework as an application-layer protocol

Security threat Scenario Analysis:

• Communication framework open to third parties: if you use Netty as an RPC framework or a private stack, the RPC framework is open to non-trusted third parties, such as opening some of the internal capabilities through the service, which requires security authentication, if the public IP is open, For some services that have very high security requirements, such as online payments, ordering, and so on, you need to communicate through SSL/TLS.

• The security of the application layer protocol. As a high-performance, asynchronous event-driven NIO framework, Netty is ideal for building application-layer protocols on the upper layer. Since the vast majority of application-level protocols are public, this means that the underlying Netty needs to provide the communication layer's secure transport capabilities to the upper layers.

SSL/TLS

Netty Secure Transmission characteristics:

• Supports SSL V2 and V3

• Support for TLS

• Support SSL One-way authentication, two-way authentication and third-party CA authentication.

The SSL one-way authentication flowchart is as follows:

Netty provides support for SSL through Sslhandler, which includes SSL V2, SSL V3, and TLS for SSL protocol types.

1. One-way authentication: One-way authentication, that is, the client only verifies the legality of the service side, the server does not authenticate the client.

2. Two-way authentication: Unlike one-way authentication, the client is also required to be securely authenticated by the server. This means that the client's self-signed certificate also needs to be imported into the digital certificate repository on the server side.

3. CA certification: Self-signed SSL bidirectional authentication, as long as the client or server to modify the key and certificate, you need to re-signing and certificate exchange, this debugging and maintenance workload is very large. Therefore, a third-party CA certification Authority is often used for signing and validating in real-world commercial systems. Our browser saves a few common ca_root. Each time you connect to a website, the certificate for this site is signed by these ca_root. You can pass the verification.

Scalable security Features

With the extended nature of Netty, you can customize your security policy:

• IP address blacklist mechanism

• Access authentication

• Sensitive information encryption or filtering mechanism

IP address blacklist is a relatively common weak security policy, it is the service side in the process of communication with the client, the client's IP address is verified, if the other IP is found in the blacklist list, then refuse to communicate with, close the link.

Access authentication policy is very many, usually strong security authentication policy, for example, based on user name + password Authentication, authentication content is often used in encryption, such as Base64+aes.

Netty the extensibility of architectural anatomy

With the extended nature of Netty, you can customize your security policy:

• The threading model can be extended

• Serialization Mode extensible

• The upper stack is extensible

• Provides a large number of network event slices to facilitate user function expansion

Netty's architectural scalability design concept is as follows:

• Determine the extension point, pre-reserved the relevant extension interface, to the user two times custom and extended use;

• The main function points are based on interface programming, which is convenient for user customization and expansion.

Best answers

Q: As far as I know, the implementation of the Java NIO selector underlying windows is a two random port interconnect to monitor connection or read-write events, and is implemented on Linux using pipelines; I have encountered such requirements, need to occupy a lot of fixed port to do the service side, What is the best solution if, under Windows, the use of the NIO framework (Mina or Netty) is likely to cause port conflicts?

You say the problem does exist, Linux uses pipe to implement network monitoring, Windows to start the port. There is no better way, the recommended way is to plan a scope as a port on the server, and then dynamically generate based on node and process information, and if a port conflict is found, you can regenerate a new port based on the algorithm within the planning scope.

Q: Please, I am now integrating Spring with Netty, using Spring's service to turn on the Netty main thread, but the Netty TCP server port will not be released when the entire container is stopped. When exiting processing, is there any good way to release the Netty server port?

In fact, it doesn't matter who pulls up the Netty main thread. What we need to do is that when the application container exits (Spring context destroys), the port and NiO thread resources are freed by invoking the graceful exit interface of Netty before exiting. Please refer to this article: Http://www.infoq.com/cn/articles/netty-elegant-exit-mechanism-and-principles

Q: Is it appropriate to write Web communications with Netty?

Netty is not a web framework, cannot parse JSP, HTML, JS, etc., but it can do web communication, for example, you can use Netty to rewrite the HTTP/HTTPS communication protocol stack of Tomcat.

Q: Can you explain the Netty serial non-locking design, how to achieve the best in serial and parallel?

To maximize performance, the Netty uses a serial, non-locking design that performs serial operations within the IO thread to avoid performance degradation caused by multi-threaded contention. On the surface, the serialization design seems to be low CPU utilization and not enough concurrency. However, by adjusting the thread parameters of the NIO thread pool, you can simultaneously start multiple serialized threads running concurrently, a locally unlocked serial threading design that performs better than one queue-multiple worker threading models. After the Netty Nioeventloop reads the message, it calls the Channelpipeline Firechannelread (Object msg) directly, as long as the user does not actively switch threads, Will always be called by the Nioeventloop to the user's handler, the period of no thread switching, this serialization process to avoid the multi-threaded operation caused by the competition of the lock, from the performance point of view is optimal.

The way to read Netty's high-performance architecture