Netty Series (four) TCP unpacking and sticky packets

Netty Series (iv) TCP unpacking and gluing one, unpacking, and sticky pack issues

(1) A small socket buffer problem

In a stream-based transmission, such as TCP/IP, the received data is stored in a socket receive buffer first. Unfortunately, stream-based transmissions are not a packet queue, but a byte queue. Even if you send 2 separate packets, the operating system will not be processed as 2 messages but only as a series of bytes. Therefore, there is no guarantee that the data you write remotely will be read accurately. For example, let's assume that the operating system's TCP/TP protocol stack has received 3 packets:

Due to this general nature of the stream-based protocol, there is a high probability that the data will be read into the following fragments in your application.

Therefore, a receiver, regardless of whether he is a client or a server, should organize the received data into one or more interesting data that can be better understood by the business logic of the program. In the example above, the received data should be constructed in the following format:

Test:

1. Send three data to server side on client side

   //向服务器发送数据 buff.channel().writeAndFlush(Unpooled.copiedBuffer("ABC".getBytes()));f.channel().writeAndFlush(Unpooled.copiedBuffer("DEF".getBytes()));f.channel().writeAndFlush(Unpooled.copiedBuffer("GHI".getBytes()));

2. The server side may treat three transmitted data as a single request, and the servers receive the following results

   ABCDEFGHI

(2) Solutions

Solutions for unpacking and sticking problems, according to industry mainstream agreements, in three scenarios, the first three Netty have been implemented:

1. Message length, such as the size of each message is fixed to 200 bytes, if not enough, empty space to fill.

2. Add special characters at the end of the package to split, such as add carriage returns.

3. Divides the message into a fixed-length message header and a message body, contains a field representing the total message length in the message header, and then handles the business logic. The first field of a message header is typically designed to use Int32 to represent the total length of a message.

Second, the fixed length scheme-fixedlengthframedecoder

The Fixedlengthframedecoder is a fixed-length decoder that automatically decodes messages at a specified length, and is useful for developers who do not need to consider TCP's sticky/unpacking problems. Note: The length is not enough to ignore.

The function of Stringdecoder is simply to convert the received object to a string, and then continue to invoke the subsequent Handler. The Fixedlengthframedecoder + Stringdecoder combination is decoded by a fixed-length text.

2. Add the following configuration to the Server:

   childHandler(new ChannelInitializer<SocketChannel>() {    @Override    public void initChannel(SocketChannel sc) throws Exception {        //定长拆包：5个字符，不足5位则忽略        sc.pipeline().addLast(new FixedLengthFrameDecoder(5));        //设置字符串形式的解码        sc.pipeline().addLast(new StringDecoder());        sc.pipeline().addLast(new ServerHandler());    }})

4. Receive the requested data in Serverhandler:

   public void channelRead(ChannelHandlerContext ctx, Object msg) {    System.out.println((String)msg);    //写给客户端    ChannelFuture f = ctx.writeAndFlush(Unpooled.copiedBuffer(((String)msg).getBytes()));    //写完成后会自动关闭客户端    //f.addListener(ChannelFutureListener.CLOSE);}

6. Data sent by the Client:

   //向服务器发送数据 buff.channel().writeAndFlush(Unpooled.copiedBuffer("aaaaabbbbb".getBytes()));f.channel().writeAndFlush(Unpooled.copiedBuffer("cccccddd".getBytes()));

8. As a result, you can see 5 characters as a request processing, less than 5 bits of ignore:

   aaaaabbbbbccccc

Three, fixed delimiter scheme-Delimiterbasedframedecoder

Linebasedframedecoder works by traversing the readable bytes in the bytebuf in turn, judging if there is a "\ n" or "\ r \ n", and if so, the end position of this position, the byte from the readable index to the end of the range is composed of one row. It is a decoder that ends with a newline character, supports either a carry terminator or no terminator, and supports the maximum length of the configuration line. If a newline character is not found after continuous reading to the maximum length, an exception is thrown, ignoring the previously read exception stream.

The

Delimiterbasedframedecoder automatically decodes messages that are identified with a delimiter as the end of the stream.

1. Add the following configuration to the Server:

   childHandler(new ChannelInitializer<SocketChannel>() {    @Override    public void initChannel(SocketChannel ch) throws Exception {        ByteBuf buf = Unpooled.copiedBuffer("$_".getBytes());        ch.pipeline().addLast(new DelimiterBasedFrameDecoder(1024, buf));        //ch.pipeline().addLast(new LineBasedFrameDecoder(1024, buf));        //设置字符串形式的解码        ch.pipeline().addLast(new StringDecoder());        ch.pipeline().addLast(new ServerHandler());    }})

2. Receive the requested data in Serverhandler:

   public void channelRead(ChannelHandlerContext ctx, Object msg) {    System.out.println((String)msg);    //写给客户端    ChannelFuture f = ctx.writeAndFlush(Unpooled.copiedBuffer("netty$_".getBytes()));    //写完成后会自动关闭客户端    f.addListener(ChannelFutureListener.CLOSE);}

   As a result, you can see that the request was processed three times:

   ABCDEFGHI

   Iv. Custom Protocols

Netty The custom protocol please refer to this article

Reference:

"Netty solve the problem of TCP unpacking sticky packet": http://ifeve.com/netty5-user-guide/#%E6%B5%81%E6%95%B0%E6%8D%AE%E7%9A%84%E4%BC%A0%E8%BE%93% E5%a4%84%e7%90%86

Record a little bit every day. Content may not be important, but habits are important!

Netty Series (four) TCP unpacking and sticky packets