Simple RPC framework-heartbeat and reconnection mechanism, rpc framework heartbeat

Simple RPC framework-heartbeat and reconnection mechanism, rpc framework heartbeat
Heartbeat

It is to tell others that they are still alive. In the simple RPC framework, TCP persistent connections are used. To ensure a valid persistent connection, a notification mechanism between the client and the server is required to inform the other party of the survival status.

How to Implement

The client sends a heartbeat message.

When the status is idle, the system periodically sends the PING message to the server.

The server receives heartbeat messages.

Capture events in idle status of the Channel. If you receive PING messages from the client, send the PONG message to the server. If the PING message of the client is not received within a certain period of time, it indicates that the client is no longer online and the channel is closed.

Client Management available connections

Because the server closes the channel because it cannot receive the PING message from the server for a long time, the connection availability of the cached client changes. You need to remove the unavailable ones from the available list and process the unavailable connections, such as dropping them directly or reconnecting them.

Prerequisites

The relationship between ChannelPipeline and handle. The handle in netty is similar to the filter function in spring mvc. ChannelPipeline can be understood as a handle container, which can be registered with a number of event processors that process different business functions, such:

• Encoding
• Decoding
• Heartbeat
• Permission
• Encryption
• Decryption
• Business Code Execution
• ......

Implement idle Processor

You can use the IdleStateHandler provided by netty to send the PING-PONG message. This processor mainly captures channel timeout events, mainly including three types

• Read timeout. No data is read from the channel within a certain period of time.
• Write timeout. No data is written from the channel within a certain period of time.
• Read/write timeout. No data is read from or written to the channel within a certain period of time.

Add Idle State processor to the client

When the client detects read/write timeout, it sends a PING message to the server if the event is triggered.

Server added idle Processor

The server only needs to capture the read timeout. When the read timeout is triggered, the channel is closed.

Why do I send heartbeat messages when I am idle?

When a normal client interacts with the server, it means that both parties are working normally and no additional heartbeat is required to inform the other party of the survival. The heartbeat message is used to detect the survival of the recipient only when the recipient does not receive the message within a certain period of time. This is also a way to improve efficiency.

Abstract heartbeat Processor

Create AbstractHeartbeatHandler and inherit the ChannelInboundHandlerAdapter, which serves the heartbeat processor of the client and the server. Determine the Message Type in the read method:

• Send the PONG message to the client if it is a PING message.
• If you receive a PONG message, directly print the message indicating that the client has successfully received the PONG message returned by the server.
• If it is another type of message, it notifies the next processor to process the message.

public void channelRead(ChannelHandlerContext channelHandlerContext, Object msg) throws Exception {        if(!(msg instanceof RpcMessage)){            channelHandlerContext.fireChannelRead(msg);            return;        }        RpcMessage message=(RpcMessage)msg;        if(null==message||null==message.getMessageHeader()){            channelHandlerContext.fireChannelRead(msg);            return;        }        if(message.getMessageHeader().getType()== Constants.MESSAGE_TYPE_HEARTBEAT_PONG){            logger.info("ClientHeartbeatHandler.channelRead0 ,pong data is:{}",message.getMessageBody());        }        else if(message.getMessageHeader().getType()== Constants.MESSAGE_TYPE_HEARTBEAT_PING){            this.sendPong(channelHandlerContext);        }        else {            channelHandlerContext.fireChannelRead(msg);        }    }

Idle State events can be processed according to different States, and three rewrite events are defined for the client and server processor to confirm and process the events.

public void userEventTriggered(ChannelHandlerContext ctx, Object evt) throws Exception {        if (evt instanceof IdleStateEvent) {            IdleStateEvent e = (IdleStateEvent) evt;            switch (e.state()) {                case READER_IDLE:                    this.handleReaderIdle(ctx);                    break;                case WRITER_IDLE:                    this.handleWriterIdle(ctx);                    break;                case ALL_IDLE:                    this.handleAllIdle(ctx);                    break;                default:                    break;            }        }    }

Client heartbeat Processor

Inherit the abstract heartbeat processor and rewrite the event to send the PING message.

public class ClientHeartbeatHandler extends AbstractHeartbeatHandler {    @Override    protected void handleAllIdle(ChannelHandlerContext ctx) {        this.sendPing(ctx);    }}

Server heartbeat Processor

Inherit the abstract heartbeat processor and rewrite the event to close the channel.

public class ServerHeartbeatHandler extends AbstractHeartbeatHandler {    @Override    protected void handleReaderIdle(ChannelHandlerContext ctx) {        logger.info("ServerHeartbeatHandler.handleReaderIdle reader timeout ,close channel");        ctx.close();    }}

Added the heartbeat processor to the ChannelPipeline client.

For example, a timeout event is triggered if no data is written in five seconds or the channel data is read.

.addLast(new IdleStateHandler(0, 0, Constants.ALLIDLE_TIME_SECONDS));

Heartbeat processor added to ChannelPipeline on the server

For example, if a channel message is not received within 10 seconds, the read timeout event is triggered.

 .addLast(new IdleStateHandler(Constants.READER_TIME_SECONDS, 0, 0))

Client message example

Normally, the heartbeat message is displayed as shown in. The message content can be customized according to your own situation.

Client offline message example

Stop the client program, trigger the read timeout event on the server, and disable the channel.

Client available connection management

Due to the aforementioned server heartbeat processor, the communication pipeline is closed after the read timeout is triggered, which causes the connection status cached by the client to become unavailable, to enable the client to get only available connections, you need to determine the status of the connection obtained from the cache. If the connection is available, the client will return directly, if not, delete the connection from the available list and take the next available connection.

Modify the connection obtaining Method

The isActive attribute of the channel can be used to determine whether the connection is available. If the connection cannot be deleted or retrieved again.

public RpcClientInvoker getInvoker() {        // ...        int index = loadbalanceService.index(size);        RpcClientInvoker invoker= RpcClientInvokerCache.get(index);        if(invoker.getChannel().isActive()) {            return invoker;        }        else {            RpcClientInvokerCache.removeHandler(invoker);            logger.info("invoker is not active,so remove it and get next one");            return this.getInvoker();        }    }

Background startup Task Processing unavailable connection

Start a thread that executes a task every five seconds, regularly retrieve unavailable connections, reconnect, and delete the unavailable connections.

Here, the reconnection process directly discards the original unavailable connection and creates a new connection.

    private static final Logger logger = LoggerFactory.getLogger(RpcClientInvokerManager.class);    static {        executorService.schedule(new Runnable() {            @Override            public void run() {                while (true) {                    List<RpcClientInvoker> notConnectedHandlers = RpcClientInvokerCache.getNotConnectedHandlers();                    if (!CollectionUtils.isEmpty(notConnectedHandlers)) {                        for (RpcClientInvoker invoker : notConnectedHandlers) {                            RpcClientInvokerManager.getInstance(referenceConfig).connect();                        }                        RpcClientInvokerCache.clearNotConnectedHandler();                    }                }            }        }, Constants.RECONNECT_TIME_SECONDS,TimeUnit.SECONDS);    }

Source code

Https://github.com/jiangmin168168/jim-framework

The code in this article depends on the above project. If you do not understand it, you can download the source code.

Reference

The figure in this article is taken from the grid