The way of Netty high performance architecture

Netty is a high-performance, asynchronous event-driven NIO framework that provides support for TCP, UDP, and file transfer, and as an asynchronous NIO framework, Netty All IO operations are asynchronous and non-blocking, through the future-listener mechanism, The user can take the initiative to obtain or obtain the IO operation result by the notification mechanism conveniently.

As the most popular NIO framework, Netty has been widely used in the field of Internet, large data distributed computing, gaming industry and communication industry, and some well-known open source components are also based on the Netty NIO framework.

Why Choose Netty

Netty is one of the industry's most popular NIO frameworks, and its robustness, functionality, performance, customization, and scalability are all the same in similar frameworks, and it has been validated by hundreds of commercial projects, such as the RPC framework for Hadoop Avro using Netty as the underlying communication framework Many other industry-leading RPC frameworks also use Netty to build high-performance asynchronous communication capabilities.

Through the analysis of Netty, we will summarize its advantages as follows:

API simple to use, low development threshold;

Powerful, preset a variety of codec functions, support a variety of mainstream protocols;

The customization ability is strong, can expand the communication frame flexibly through Channelhandler;

With high performance, the Netty is optimal in comparison with other major NIO frameworks in the industry.

Mature, stable, netty fixes all the JDK NiO bugs that have been discovered, and business developers don't need to worry about nio bugs anymore;

Community active, version iteration cycle short, found bugs can be repaired in time, while more new features will be added;

Has experienced a large-scale commercial application test, the quality has been validated. In the Internet, large data, network games, enterprise applications, telecommunications software and many other industries have been successful business, proved that it has been fully able to meet the commercial applications of different industries.

 Netty Architecture Analysis

Netty is designed using a more typical three-tier network architecture.

First layer: Reactor communication dispatch layer, which is completed by a series of auxiliary classes, including reactor thread Nioeventloop and its parent class, Niosocketchannel/nioserversocketchannel, and its parent class, Bytebuffer and the various kinds of Buffer, Unsafe and its derived from the various internal classes. The primary responsibility for this layer is to listen for read-write and connection operations on the network, responsible for reading data from the network layer into the memory buffer, and then triggering various network events, such as connection creation, connection activation, read events, write events, and so on, triggering these events into PipeLine, a chain of responsibilities that PipeLine acts as for subsequent processing.

Second level: Responsibility chain PipeLine, it is responsible for the orderly dissemination of events in the responsibility chain, while responsible for the dynamic layout of the responsibility chain, the responsibility chain can choose to listen and handle events of their own concern, it can intercept processing and backward/forward propagation events, different applications of the Handler node functions are different, usually , often develop codec Hanlder used for message encoding and decoding, it can transform the external protocol message into internal POJO object, so that the upper business side only need to care about processing business logic, do not need to perceive the underlying protocol differences and threading model differences, to achieve the architecture level of hierarchical isolation.

The third layer: business logic processing layer, can be divided into two categories:

Pure business logic processing, such as order Processing.

Application layer protocol management, such as HTTP protocol, FTP protocol, etc.

Next, I talk about the architecture of Netty from three aspects that affect communication performance (I/O model, thread scheduling model, serialization mode). 

I/O model

There are many drawbacks to it:

Performance problem: The connection of a single thread model causes the concurrent access number and system throughput of the server to be greatly restricted;

Reliability problem: Due to the synchronous blocking mode of I/O operation, when the network congestion or communication to the end of slow processing will cause I/O thread is suspended, blocking time unpredictable;

Maintainability problem: The number of I/O threads is not effective control, resources can not be effectively shared (multithreading concurrency problem), System maintainability is poor;

Functional and characteristic comparisons of several I/O models:

The Netty I/O model is based on non-blocking I/O implementations, and the underlying dependencies are selector of the JDK NIO framework.

Selector provides the ability to choose a task that is already ready. In simple terms, selector will constantly poll the channel registered on it, and if a channel has a new TCP connection, read and write events, the channel is in a ready state and will be polled by selector. The Selectionkey can then be used to obtain a set of ready channel for subsequent I/O operations.

A multiplexer selector can poll multiple channel at the same time, because the Jdk1.5_update10 version (+) uses Epoll () instead of the traditional select implementation, so it does not have the maximum connection handle 1024/2048 limit. This means that only one thread is responsible for selector polling and can access tens of thousands of clients, which is really a huge technological advance.

After using the non-blocking I/O model, Netty solves the performance, throughput, and reliability problems associated with traditional synchronous blocking I/O. 

Thread Scheduling model

There are three types of commonly used reactor threading models, respectively, as follows:

Reactor single-threaded Model: Reactor single-threaded model, which means that all I/O operations are done on the same NIO thread. For some small-capacity scenarios, you can use a single-threaded model.

Reactor multithreaded Model: The biggest difference between Rector multithreaded models and single-threaded models is that there is a set of NIO threads that handle I/O operations. Mainly used for high concurrency, large volume of business scenarios.

Master-Slave reactor multithreaded model: the main character of the master-slave reactor threading model is that the server-side is no longer a 1 separate NIO thread for receiving client connections, but rather a separate NIO thread pool. The master-slave NIO threading model solves the performance problem of 1 server-side listener threads not being able to handle all client connections effectively.

In fact, the Netty threading model is not fixed, and the three reactor threading models can be supported by creating different Eventloopgroup instances in the Startup helper class and by using the appropriate parameter configuration.

In most scenarios, parallel multithreading can enhance the concurrency performance of the system. However, improper handling of concurrent access to shared resources can lead to severe lock competition, which ultimately leads to performance degradation. In order to avoid the performance loss of lock competition as far as possible, through the serialization design, that is, the message processing as far as possible in the same thread to complete, without the thread switching, so as to avoid multithreading competition and synchronization lock.

In order to improve performance as much as possible, Netty uses a serial, non-disruptive design to perform serial operations within I/O threads to avoid the performance degradation caused by multi-threaded competition. On the surface, the serialization design seems to be low in CPU utilization and less concurrent. However, by adjusting the thread parameters of the NIO thread pool, you can simultaneously start multiple serialized threads running concurrently, and this locally unlocked serial thread design is better than a queue-multiple worker threading model performance.

Serialization mode

The key factors that affect serialization performance are summarized as follows:

Serialized code stream size (network bandwidth occupancy)

Serialization & Deserialization Performance (CPU resource footprint)

Performance of concurrent calls: stability, linear growth, occasional Shiyanmao thorn, etc.

The Java serialization and binary encoding performance test, coding 1 million times, the test results show that: Java serialization performance only binary encoding of about 6.17%.

Netty default provides support for Google Protobuf, by extending the Netty codec interface, users can implement other high-performance serialization frameworks, such as the thrift Compression binary codec framework.

Different scenarios have different requirements for the serialization framework, and for high-performance scenario Netty, Google's Protobuf binary serialization framework is provided by default and can be extended based on the codec framework provided by Netty if the user has a need for other binary serialization frameworks. Reliability of Netty Architecture analysis

The reliability challenges faced by Netty:

As the underlying network communication framework for the RPC framework, failure will result in the inability to make remote service (interface) calls.

As the basic communication Framework for Application layer protocol, failure will result in the application stack not working properly.

The network environment is complex (for example, the Gsm/3g/wifi network of hand tour or push service), the trouble is unavoidable, the business cannot be interrupted.

From the application scenario, Netty is the basic communication framework, and when a bug occurs, the light needs to restart the application, and heavy may cause the entire business to break. Its reliability will affect the entire business cluster data communication and exchange, in today's distributed-oriented software architecture system, communication interruption means that the entire business interruption, the distributed architecture of communication reliability requirements are very high.

From the operating environment, Netty will face a bad network environment, which requires its own reliability to be good enough, the platform can solve the reliability problem needs to be resolved by the Netty itself, otherwise it will cause the top users to pay attention to too much of the underlying fault, which will reduce the netty ease of use, At the same time increase the user's development and operational costs.

The reliability of Netty is so important that any failure of it can cause business disruption and incur huge economic losses. As a result, Netty continuously adds new reliability features to the iteration to meet the growing demands of users for their high reliability and robustness.

Link Validity detection

The heartbeat detection mechanism provided by Netty is divided into three types:

Read idle, link duration t not read any messages;

Write idle, link duration t send no messages;

Read-write idle, link duration t does not receive or send any messages.

When the network occurs a single pass, the connection is hang by the firewall, the long time GC or the communication thread has unexpected exception, it will cause the link is not available and not easy to be discovered in time. In particular, anomalies occur in the early morning business trough, when the morning business peak arrival, because the link is not available can lead to a sudden large volume business failure or timeout, which will have a significant threat to the reliability of the system.

From the technical level, to solve the problem of link reliability, we must periodically check the validity of the link. The most popular and common practice is heartbeat detection.

Heartbeat detection mechanism is divided into three levels:

The TCP level heartbeat detection, namely TCP's keep-alive mechanism, its scope is the entire TCP protocol stack;

The heartbeat detection of the protocol layer mainly exists in the long connection protocol. such as the SMPP Agreement;

Application layer of heartbeat detection, it is mainly by the business products through the agreed way to send the heartbeat message to the other side of the implementation.

The purpose of heartbeat detection is to confirm that the current link is available, that the other person is alive and can receive and send messages normally. As a highly reliable NIO framework, Netty also provides a heartbeat detection mechanism based on link idle:

Read idle, link duration t not read any messages;

Write idle, link duration t send no messages;

Read-write idle, link duration t does not receive or send any messages.

Flow shaping

Flow shaping (traffic shaping) is a method of actively adjusting the output rate of flow. Netty's flow shaping has two functions:

Preventing the downstream network element from being crushed due to uneven performance of the upper and lower network elements, and the business process interruption;

Prevent the communication module from receiving the message too fast, the back-end business thread processing is not timely caused by the "death" problem.

Flow shaping (traffic shaping) is a method of actively adjusting the output rate of flow. A typical application is to control the output of local traffic based on the TP index of downstream network nodes. The main difference between traffic shaping and traffic regulation is that traffic shaping caches the packets that need to be discarded in the traffic regulation-usually put them in a buffer or queue, also known as traffic shaping (traffic shaping, referred to as TS). When the token bucket has enough tokens, the cached packets are sent out evenly. Another difference between traffic shaping and flow regulation is that the cosmetic may increase latency, while regulation hardly introduces additional delays.

Netty supports two types of traffic shaping modes:

Global traffic Shaping: the scope of global traffic shaping is process-level, regardless of how many channel you create, its scope is for all channel. The user can set through the parameter: the receiving rate of the message, the sending rate of the message, the shaping cycle.

Link-level Flow shaping: Single link flow shaping and global traffic shaping the biggest difference is that it is a single link for the scope of different links can be set different shaping strategy.

Graceful Downtime

Netty's Elegant Downtime trilogy:

No longer receive new messages

Preprocessing actions before exiting

Resource release operations

Java graceful downtime is usually achieved by registering the JDK's Shutdownhook, when the system receives an exit instruction, the system first marks the exit state, no longer receives new messages, then processes the backlog of messages, and finally invokes the resource recycle interface to destroy the resource, and the last thread exits execution.

Usually graceful exit requires a time-out control mechanism, such as 30S, if the arrival timeout is still not completed before the exit of the resource collection operations, and so on, by the stop script directly call kill-9 PID, Force exit.

In practical projects, Netty, as a high-performance asynchronous NIO communication framework, is often used as the basic communication framework for accessing, parsing, and scheduling various protocols. For example, in RPC and distributed service frameworks, Netty is often used as internal. The basic communication framework for proprietary protocols. When the application process gracefully exits, Netty, which is the communication framework, also needs to gracefully exit. The main reasons are as follows: release NIO threads and handles as soon as possible; if flush is used for bulk message sending, the message to be sent accumulated in the send queue needs to be accumulated. The transmission is complete; the message is being written or read, and needs to be processed; the scheduled task set in the NioEventLoop thread scheduler needs to be executed or cleaned up.

Netty architecture analysis security
The security challenges facing Netty:

Open to third parties

Basic communication framework as an application layer protocol

Analysis of security threat scenarios: Communication framework open to third parties: If Netty is used as an RPC framework or a proprietary protocol stack, the RPC framework is open to non-trusted third parties. For example, some internal capabilities are opened to the public through services. For security certification, if public IP is open, some services that require very high security, such as online payment, ordering, etc., need to communicate via SSL/TLS. Application layer protocol security. As a high-performance, asynchronous event-driven NIO framework, Netty is ideal for building upper-level application layer protocols. Since most application layer protocols are public, this means that the underlying Netty needs to provide secure transport at the communication layer to the upper layer. SSL/TLSNetty secure transmission features: 1. Support SSL V2 and V32. Support TLS3. Support SSL one-way authentication, two-way authentication and third-party CA authentication.

Netty provides support for SSL through SslHandler, which supports SSL protocol types including: SSL V2, SSL V3, and TLS. One-way authentication: One-way authentication, that is, the client only verifies the legality of the server, and the server does not verify the client. Two-way authentication: Unlike one-way authentication, the server also needs to perform security authentication on the client. This means that the client's self-signed certificate also needs to be imported into the server's digital certificate store. CA authentication: Based on self-signed SSL mutual authentication, as long as the client or server modifies the key and certificate, it needs to re-sign and exchange the certificate. This debugging and maintenance workload is very large. Therefore, in actual commercial systems, third-party CA certification authorities are often used for signing and verification. Our browser saves several commonly used CA_ROOTs. Each time you connect to the website, the certificate for this website is signed by these CA_ROOTs. It can be verified.

Scalable security features
With Netty's extended features, you can customize your security policy:

1. IP address blacklist mechanism

2. Access authentication

3. Sensitive information encryption or filtering mechanism
The IP address blacklist is a commonly used weak security protection policy. It is characterized in that the server verifies the IP address of the client during communication with the client. If the other party's IP is found in the blacklist, it refuses to Communication, close the link. There are many access authentication policies, which are usually strong security authentication policies. For example, authentication based on username and password, authentication content is often encrypted, such as Base64+AES. Here I recommend an architecture learning exchange group. Exchange learning group number: 478030634 will share some video recordings recorded by senior architects: Spring, MyBatis, Netty source code analysis, high concurrency, high performance, distributed, microservice architecture principles, JVM performance optimization, distributed architecture, etc. These become the necessary knowledge systems for architects. I also receive free learning resources and I am currently benefiting a lot.

Extendment of Netty Architecture Analysis
With Netty's extended features, you can customize your security policy:

1. Thread model can be extended

2. Serialization method can be extended

3. The upper protocol stack can be extended

4. Provide a large number of network event aspects to facilitate user function expansion

Netty's architectural scalability design philosophy is as follows:

5. Determine the extension point, reserve the relevant extension interface in advance, and give the user secondary customization and extended use;

6. The main function points are based on interface programming, which is convenient for users to customize and expand.