Netty4 Combat-TCP Sticky & unpacking Solutions

Netty is one of the most popular NIO frameworks in the industry, and its robustness, high performance, customization, and scalability are all the same in the same framework. It has been validated by hundreds of commercial projects, such as the RPC framework Avro of Hadoop, which uses Netty as the underlying communication framework, and other industry-leading RPC frameworks, such as Dubbo, Google Open source Grpc, Sina Weibo open source Motan, Twitter's Open-source Finagle also uses Netty to build high-performance asynchronous communication capabilities. In addition, Alibaba Open-source message middleware ROCKETMQ also uses Netty as the underlying communication framework. TCP Sticky pack/unpacking

TCP is a "stream" protocol, called a stream, which is a long string of binary data without boundaries. TCP as a Transport layer protocol does not understand the specific meaning of upper-level business data, it will be based on the actual situation of the TCP Buffer Packet division, so the business is considered a complete package, may be split into multiple packets TCP can be sent, but also may be a number of small packets packaged into a large packet sent, This is known as TCP sticky and unpacking issues. solution to the problem of sticky packaging

Because the bottom of TCP can not understand the upper level of business data, so at the bottom is not guarantee that packets are not split and reorganization, this problem can only be solved by the application of the upper layer of the stack design. The industry's mainstream agreement solution can be summed up as follows:
1. Message fixed length, packet size, such as the length of each message fixed to 200 bytes, if not enough empty space to fill;
2. Add special separator, such as the end of each message to add a carriage return line character (such as FTP protocol) or to specify special characters as message separators, the receiver through the special separator segmentation message distinction;
3. Divides the message into a message header and a message body, which contains a field representing the total length of the information (or message length);
4. More complex custom application layer protocols. Netty solution for sticky and unpacking packages

Netty provides a number of decoders that can be used to subcontract operations, respectively:
* Linebasedframedecoder
* Delimiterbasedframedecoder (Add special separator message to subcontract)
* Fixedlengthframedecoder (use fixed-length message to subcontract)
* Lengthfieldbasedframedecoder Linebasedframedecoder Decoder

Linebasedframedecoder is a carriage return newline decoder, and if the user sends a message with a carriage return line feed as the end of the message, you can decode the message directly using Netty Linebasedframedecoder. It is only necessary to add linebasedframedecoder to the channelpipeline when initializing the Netty server or client, without having to implement a new set of newline decoders. Netty Dependence

        <dependency>
            <groupId>io.netty</groupId>
            <artifactid>netty-all</artifactid >
            <version>4.1.9.Final</version>
        </dependency>

1.1 Server End

Import Io.netty.bootstrap.ServerBootstrap;
Import io.netty.channel.*;
Import Io.netty.channel.nio.NioEventLoopGroup;
Import Io.netty.channel.socket.SocketChannel;
Import Io.netty.channel.socket.nio.NioServerSocketChannel;
Import Io.netty.handler.codec.LineBasedFrameDecoder;
Import Io.netty.handler.codec.string.StringDecoder;
Import Io.netty.handler.codec.string.StringEncoder;
Import Org.slf4j.Logger;

Import Org.slf4j.LoggerFactory; /** * * * @author Ricky * */public class Linebasedserver {private Logger Logger = Loggerfactory.getlogger (getclass

    ());
        public void bind (int port) throws Exception {Eventloopgroup bossgroup = new Nioeventloopgroup ();
        Eventloopgroup Workergroup = new Nioeventloopgroup ();
            try {serverbootstrap b = new Serverbootstrap (); B.group (Bossgroup, Workergroup). Channel (nioserversocketchannel.class). Option (Channeloption.so_ba Cklog, 1024). Childhandler (New ChannelinitializEr<socketchannel> () {@Override public void Initchannel (Socketchannel ch) throws exc
                    eption {Channelpipeline p = ch.pipeline ();
                    P.addlast (New Linebasedframedecoder (1024));
                    P.addlast (New Stringdecoder ());

                    P.addlast (New Stringencoder ());
                P.addlast (New Lineserverhandler ());

            }
            });
            Bind and start to accept incoming connections. Channelfuture f = b.bind (port). sync ();

            (7) Logger.info ("Server Bind port:{}", port);
            Wait until the "server socket is closed."

        F.channel (). Closefuture (). sync ();
            finally {bossgroup.shutdowngracefully ();
        Workergroup.shutdowngracefully (); } public static void Main (string[] args) throws Exception {new Linebasedserver (). bind (Constants.port)
    ; }
}

Import Io.netty.channel.ChannelHandlerContext;
Import Io.netty.channel.ChannelInboundHandlerAdapter;
Import Org.slf4j.Logger;
Import org.slf4j.LoggerFactory;

public class Lineserverhandler extends Channelinboundhandleradapter {
    private Logger Logger = Loggerfactory.getlogger (GetClass ());

    private int count = 0;
    @Override public
    void Channelread (Channelhandlercontext ctx, Object msg)
            throws Exception {

        count++;
        String BODY = (string) msg;
        Logger.info ("Server Read msg:{}, count:{}", body, count);

        String response = "Hello from server" +system.getproperty ("Line.separator");
        Ctx.writeandflush (response);
    }

    @Override public
    void Exceptioncaught (Channelhandlercontext ctx, throwable cause)
            throws Exception {
        Logger.error ("Server caught Exception", cause);
        Ctx.close ();
    }

1.2 Client

Import Io.netty.bootstrap.Bootstrap;
Import io.netty.channel.*;
Import Io.netty.channel.nio.NioEventLoopGroup;
Import Io.netty.channel.socket.SocketChannel;
Import Io.netty.channel.socket.nio.NioSocketChannel;
Import Io.netty.handler.codec.LineBasedFrameDecoder;
Import Io.netty.handler.codec.string.StringDecoder;

Import Io.netty.handler.codec.string.StringEncoder; public class Linebasedclient {public void connect (String host, int port) throws Interruptedexception {even
        Tloopgroup Group = new Nioeventloopgroup ();
            try {Bootstrap b = new Bootstrap ();
            B.group (Group). Channel (niosocketchannel.class). Option (Channeloption.tcp_nodelay, True) . Handler (new channelinitializer<socketchannel> () {@Override protected void INITC
                    Hannel (Socketchannel ch) throws Exception {Channelpipeline p = ch.pipeline (); P.addlast (New LinebasedframedecodER (1024));
                    P.addlast (New Stringdecoder ());

                    P.addlast (New Stringencoder ());
                P.addlast (New Lineclienthandler ());

            }
            });

            Channelfuture future = B.connect (Constants.host, constants.port). sync ();
        Future.channel (). Closefuture (). sync ();
        finally {group.shutdowngracefully (); } public static void Main (string[] args) throws Exception {new Linebasedclient (). Connect (Constants.ho
    ST, Constants.port); }
}

Import Io.netty.channel.ChannelHandlerContext;
Import Io.netty.channel.ChannelInboundHandlerAdapter;
Import Org.slf4j.Logger;

Import Org.slf4j.LoggerFactory; public class Lineclienthandler extends Channelinboundhandleradapter {private Logger Logger = Loggerfactory.getlogger (

    GetClass ());

    private int count = 0; @Override public void channelactive (Channelhandlercontext ctx) throws Exception {//Send the message to Serve
           R for (int i=0; i<100; i++) {String msg = ' Hello from client ' +i;

           Logger.info ("Client Send message:{}", msg);
       Ctx.writeandflush (Msg+system.getproperty ("Line.separator"));
        @Override public void Channelread (Channelhandlercontext ctx, Object msg) throws Exception {
        String BODY = (string) msg;
        count++;
    Logger.info ("Client Read msg:{}, count:{}", body, count); @Override public void Exceptioncaught (Channelhandlercontext ctx, Throwable Cause) throws Exception {Logger.error ("Client caught Exception", cause);
    Ctx.close (); }
}

Delimiterbasedframedecoder Decoder

The Delimiterbasedframedecoder is a delimiter decoder that allows the user to specify the delimiter that ends the message, and it can automatically decode the message that is identified by the delimiter as the end of the code stream. The carriage return newline decoder is actually a special Delimiterbasedframedecoder decoder.

Import Io.netty.bootstrap.ServerBootstrap;
Import io.netty.buffer.Unpooled;
Import io.netty.channel.*;
Import Io.netty.channel.nio.NioEventLoopGroup;
Import Io.netty.channel.socket.SocketChannel;
Import Io.netty.channel.socket.nio.NioServerSocketChannel;
Import Io.netty.handler.codec.DelimiterBasedFrameDecoder;
Import Io.netty.handler.codec.string.StringDecoder;
Import Io.netty.handler.codec.string.StringEncoder;
Import Org.slf4j.Logger;

Import Org.slf4j.LoggerFactory; /** * * * @author Ricky * */public class Delimiterserver {private Logger Logger = Loggerfactory.getlogger (getclas


    s ());
        public void bind (int port) throws Exception {Eventloopgroup bossgroup = new Nioeventloopgroup (1);
        Eventloopgroup Workergroup = new Nioeventloopgroup ();
            try {serverbootstrap b = new Serverbootstrap (); B.group (Bossgroup, Workergroup). Channel (nioserversocketchannel.class). Option (Channeloption.so_ba
     Cklog, 1024)       . Childhandler (New channelinitializer<socketchannel> () {@Override public void
                    Initchannel (Socketchannel ch) throws Exception {Channelpipeline p = ch.pipeline ();
                    P.addlast (New Delimiterbasedframedecoder (1024, Unpooled.copiedbuffer (Constants.DELIMITER.getBytes ()));
                    P.addlast (New Stringdecoder ());

                    P.addlast (New Stringencoder ());
                P.addlast (New Delimiterserverhandler ());

            }
            });
            Bind and start to accept incoming connections. Channelfuture f = b.bind (port). sync ();

            (7) Logger.info ("Server Bind port:{}", port);
            Wait until the "server socket is closed."

        F.channel (). Closefuture (). sync ();
            finally {bossgroup.shutdowngracefully ();
        Workergroup.shutdowngracefully (); }} public static void Main (string[] args) Throws Exception {new Delimiterserver (). bind (Constants.port); }
}

Client:

Import Io.netty.bootstrap.Bootstrap;
Import io.netty.buffer.Unpooled;
Import io.netty.channel.*;
Import Io.netty.channel.nio.NioEventLoopGroup;
Import Io.netty.channel.socket.SocketChannel;
Import Io.netty.channel.socket.nio.NioSocketChannel;
Import Io.netty.handler.codec.DelimiterBasedFrameDecoder;
Import Io.netty.handler.codec.string.StringDecoder;

Import Io.netty.handler.codec.string.StringEncoder; public class Delimiterclient {public void connect (String host, int port) throws Interruptedexception {even
        Tloopgroup Group = new Nioeventloopgroup ();
            try {Bootstrap b = new Bootstrap ();
            B.group (Group). Channel (niosocketchannel.class). Option (Channeloption.tcp_nodelay, True) . Handler (new channelinitializer<socketchannel> () {@Override protected void INITC
               Hannel (Socketchannel ch) throws Exception {Channelpipeline p = ch.pipeline ();     P.addlast (New Delimiterbasedframedecoder (1024, Unpooled.copiedbuffer (Constants.DELIMITER.getBytes ()));
                    P.addlast (New Stringdecoder ());

                    P.addlast (New Stringencoder ());
                P.addlast (New Delimiterclienthandler ());

            }
            });

            Channelfuture future = B.connect (Constants.host, constants.port). sync ();
        Future.channel (). Closefuture (). sync ();
        finally {group.shutdowngracefully (); } public static void Main (string[] args) throws Exception {new Delimiterclient (). Connect (Constants.ho
    ST, Constants.port); }
}

Fixedlengthframedecoder Decoder

Fixedlengthframedecoder is a fixed length decoder, which can automatically decode messages according to the specified length, and developers do not need to consider TCP's sticky/unpacking problems, very practical.

For fixed-length messages, if the actual length of the message is less than the fixed length, the complement operation is often performed, which leads to the waste of space and resources to some extent. But its advantages are very obvious, codec is relatively simple, so in the actual project still has a certain application scene.

Import Io.netty.bootstrap.ServerBootstrap;
Import Io.netty.buffer.ByteBuf;
Import io.netty.buffer.Unpooled;
Import io.netty.channel.*;
Import Io.netty.channel.nio.NioEventLoopGroup;
Import Io.netty.channel.socket.SocketChannel;
Import Io.netty.channel.socket.nio.NioServerSocketChannel;
Import Io.netty.handler.codec.FixedLengthFrameDecoder;
Import Io.netty.handler.codec.string.StringDecoder;
Import Io.netty.handler.codec.string.StringEncoder;
Import Io.netty.handler.logging.LogLevel;
Import Io.netty.handler.logging.LoggingHandler;
Import Org.slf4j.Logger;

Import Org.slf4j.LoggerFactory;

    /** * @author Ricky Fung * * public class Nettyserver {private Logger Logger = Loggerfactory.getlogger (GetClass ()); public void bind (int port) throws Interruptedexception {Eventloopgroup bossgroup = new Nioeventloopgroup (1)
        ;
        Eventloopgroup Workergroup = new Nioeventloopgroup ();
      try {//configure Server startup class Serverbootstrap B = new Serverbootstrap ();      B.group (Bossgroup, Workergroup). Channel (Nioserversocketchannel.class). Opti
                    On (Channeloption.so_backlog). Handler (new Logginghandler (Loglevel.info))//configuration log output
                        . Childhandler (New channelinitializer<socketchannel> () {@Override
                            protected void Initchannel (Socketchannel ch) throws Exception {
                            Ch.pipeline (). AddLast (New Fixedlengthframedecoder (1<<5));
                            Ch.pipeline (). AddLast (New Stringdecoder ());

                            Ch.pipeline (). AddLast (New Stringencoder ());
                        Ch.pipeline (). AddLast (New Serverhandler ());

            }
                    });
            Channelfuture f = b.bind (port). sync ();
        Wait for the server to Exit F.channel (). Closefuture (). sync (); finally {//Free thread resource bossGroup.shutdowngracefully ();
        Workergroup.shutdowngracefully ();

        } private class Serverhandler extends Channelinboundhandleradapter {private int counter = 0; @Override public void channelactive (Channelhandlercontext ctx) throws Exception {super.channelactive
        (CTX);
         } @Override