Message Service Based on Netty and RabbitMQ, nettyrabbitmq message

Source: Internet
Author: User
Tags rabbitmq

Message Service Based on Netty and RabbitMQ, nettyrabbitmq message

As a high-performance Asynchronous Network Development Framework, Netty can be used as a development framework for various services.

Some time ago, a project involved real-time data collection of hardware devices. Netty was used as the collection service implementation framework, and RabbitMQ was also used as the collection service and communication message queues of various modules, the overall service architecture is shown below:

Extract the code of the Business Code and the actual protocol parsing part to obtain the above simple design diagram. The code is open-source on GitHub, which briefly introduces several key technical points involved in the NettyMQServer collection service:

1. device TCP Message Parsing:

TCP communication is used between NettyMQServer and the Device of the collection Device. For TCP Message Parsing, you can use LengthFieldBasedFrameDecoder (message header and message body) to effectively solve the TCP Message "sticky packet" problem.

The message package parsing diagram is as follows:

 lengthFieldOffset   =  0 lengthFieldLength   =  2 lengthAdjustment    = -2 (= the length of the Length field) initialBytesToStrip =  0 BEFORE DECODE (14 bytes)         AFTER DECODE (14 bytes) +--------+----------------+      +--------+----------------+ | Length | Actual Content |----->| Length | Actual Content | | 0x000E | "HELLO, WORLD" |      | 0x000E | "HELLO, WORLD" | +--------+----------------+      +--------+----------------+

In the code, the message length is stored in 4 bytes and decoded using LengthFieldBasedFrameDecoder (65535,0, 4, netty obtains the message length from the first 4 bytes of the received data, and then obtains a TCP Message packet.

2. send messages to devices:

First, keep the TCP connection when creating the connection:

static final ChannelGroup channels = new DefaultChannelGroup(            GlobalEventExecutor.INSTANCE);    @Override    public void channelActive(ChannelHandlerContext ctx) throws Exception {        // A closed channel will be removed from ChannelGroup automatically        channels.add(ctx.channel());    }

When a Channel is Active (the connection is created), use ChannelGroup to save the Channel connection. When sending messages to a device, you can traverse the Channel group and find the corresponding Channel, send messages to the Channel:

for (io.netty.channel.Channel c : EchoServerHandler.channels) {                            ByteBuf msg = Unpooled.copiedBuffer(message.getBytes());                            c.writeAndFlush(msg);                        }

All connected devices are sent here. When the connection is disconnected, ChannelGroup automatically removes the connection and does not need to be manually managed.

3. Heartbeat Detection

When a Device fails to collect data due to power failure or other reasons, the Netty server needs to know whether the Device is working properly. You can use Netty's IdleStateHandler. The sample code is as follows:

// 3 minutes for read idlech.pipeline().addLast(new IdleStateHandler(3*60,0,0));ch.pipeline().addLast(new HeartBeatHandler());/** * Handler implementation for heart beating. */public class HeartBeatHandler extends ChannelInboundHandlerAdapter{    @Override    public void userEventTriggered(ChannelHandlerContext ctx, Object evt)            throws Exception {        if (evt instanceof IdleStateEvent) {            IdleStateEvent event = (IdleStateEvent) evt;            if (event.state() == IdleState.READER_IDLE) {                // Read timeout                System.out.println("READER_IDLE: read timeout from "+ctx.channel().remoteAddress());                //ctx.disconnect(); //Channel disconnect            }        }    }}

If no data is read for three minutes, a READER_IDLE event is triggered.

4. receive and send RabbitMQ messages

The NettyMQServer uses Spring AMQP for message sending. You only need to configure it in the configuration file for ease of use.

Spring AMQP can also be used for receiving messages from NettyMQServer. However, due to lack of familiarity with Spring-related configurations, RabbitMQ Client Java API is used for more flexible use of MQ:

                    Connection connection = connnectionFactory.newConnection();                    Channel channel = connection.createChannel();                    channel.exchangeDeclare(exchangeName, "direct", true, false, null);                    channel.queueDeclare(queueName, true, false, false, null);                    channel.queueBind(queueName, exchangeName, routeKey);                    // process the message one by one                    channel.basicQos(1);                    QueueingConsumer queueingConsumer = new QueueingConsumer(channel);                    // auto-ack is false                    channel.basicConsume(queueName, false, queueingConsumer);                    while (true) {                        QueueingConsumer.Delivery delivery = queueingConsumer.nextDelivery();                        String message = new String(delivery.getBody());                        log.debug("Mq Receiver get message");                        // Send the message to all connected clients                        // If you want to send to a specified client, just add                        // your own logic and ack manually                        // Be aware that ChannelGroup is thread safe                        log.info(String.format("Conneted client number: %d",EchoServerHandler.channels.size()));                        for (io.netty.channel.Channel c : EchoServerHandler.channels) {                            ByteBuf msg = Unpooled.copiedBuffer(message.getBytes());                            c.writeAndFlush(msg);                        }                        // manually ack to MQ server the message is consumed.                        channel.basicAck(delivery.getEnvelope().getDeliveryTag(), false);
                    }

The above Code reads data from a Queue. To effectively process data and prevent abnormal data loss, manual Ack is used.

RabbitMQ usage: http://www.cnblogs.com/luxiaoxun/p/3918054.html

 

Code hosted on GitHub: https://github.com/luxiaoxun/NettyMqServer

 

Refer:

Http://netty.io/

Http://netty.io/4.0/api/io/netty/handler/codec/LengthFieldBasedFrameDecoder.html

Http://netty.io/4.0/api/io/netty/handler/timeout/IdleStateHandler.html

 

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: info-contact@alibabacloud.com and provide relevant evidence. A staff member will contact you within 5 working days.

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.