A read incomplete bug fix record that occurs when a large data volume message is received using the netty4.x client

1, first say the problem

Background: The service is provided by a security gateway running on Linux, the TCP protocol sends a binary encoded XML string message, the first field of the header is the int type representing the byte-order tag, and the second field is the int type representing the entire message length.

Phenomenon: When the data volume is small, it can send and receive packets normally, when the service side sends a large amount of packet data (this example is nearly 600k) the probability of receiving data directly call Readcomplete () method without walking Channelread ()

Trace: The trace code found the problem when the read () method of the context reads to more than 100 K (sometimes more than 200 may also be more than 300, in short, not yet read all the data) when the data read should have been 1024 bytes (fill the default allocation of BYTEBUF) but read only 576 bytes;

The Netty framework code determines that if the number of bytes currently read is less than BYTEBUF, the size is considered to be read completion, so the Readcomplete () method is called and an error occurs ...

Solution: Add a flag flag to the ClientHandler class to determine whether the data is read normally. The Channelread () method normally calls the True;readcomplete method to add a judgment that only if the flag is true closes the context otherwise it continues to call Ctx.read ().

2, then the core code

Client:

1ClientHandler ClientHandler =NewClientHandler ( This);2 Bootstrap.group (EventLoop)3. Channel (Niosocketchannel.class)4. Option (Channeloption.tcp_nodelay,true)5. option (Channeloption.connect_timeout_millis, 3000)6 . Option (Channeloption.max_messages_per_read, Integer.max_value)7. Handler (NewClientchannelinitializer (ClientHandler));8             9Channelfuture f =Bootstrap.connect (host, port). sync ();Ten  OneF.channel (). Closefuture (). sync ();

1 Private classClientchannelinitializerextendsChannelinitializer<socketchannel> {2         PrivateClientHandler ClientHandler;3 4          PublicClientchannelinitializer (ClientHandler clienthandler) {5              This. ClientHandler =ClientHandler;6         }7 8 @Override9         protected voidInitchannel (Socketchannel Socketchannel)throwsException {Ten  OneSocketchannel.pipeline (). AddLast (NewSpldecoder ()); A socketchannel.pipeline (). AddLast (ClientHandler); -Channel =Socketchannel; -         } the}

The implementation of the Initchannel before solving the problem is this, using the Netty internal length field decoder
@Overrideprotectedvoidthrows  Exception {    ch.pipeline (). AddLast ( New Lengthfieldbasedframedecoder (integer.max_value,4,4,-8,0));    Ch.pipeline (). AddLast (ClientHandler);}

ClientHandler:

 Public class extends Channelinboundhandleradapter {

1 @Override2      Public voidChannelactive (Channelhandlercontext context)throwsException {3Logger.info ("Ready to send Request ...");4Bytebuffer result =Getbytebuffer ();5Bytebuf buf =Unpooled.buffer (result.remaining ());6 buf.writebytes (result);7 8 Context.writeandflush (BUF);9     }Ten  One @Override A      Public voidChannelread (Channelhandlercontext context, Object msg)throwsException { -Logger.info ("Get Server response ..."); -  theString[] result =(string[]) msg; -  -Logger.debug ("Response XML is:" + result[1]); - client.setresponse (result); +  -OK =true; +     } A  at @Override -      Public voidChannelreadcomplete (Channelhandlercontext ctx)throwsException { - Ctx.flush (); -         if(OK) { - ctx.close (); -}Else { in Ctx.read (); -         } to}

3, finally said the solution process

At first I suspected that it was caused by improper use of Netty's fixed-length field decoder lengthfieldbasedframedecoder parameter, because I thought it didn't understand it deeply. and wrote a decoder that inherits Bytetomessagedecoder can solve the problem of unpacking and decoding function, but the problem is still the probability of appearing ...

Then took a try attitude in the ClientHandler added an instance property OK (default false), after the normal execution of the Channelread () method is set to True,readcomplete () method to make a judgment if ok== False call Ctx.read (), run discovery to solve the problem perfectly

Because the call to read () method is to continue reading the data instead of re-reading (because the CTX and channel, pipline and other data states have not changed)!

In the Spldecoder class, add the current read data to print information: "Logger.debug (" read data: This time "+ Readablebytes +"; cumulative "+ Currentlength +"; total); " ；

Add print error message in front of Ctx.read () "****** read data incomplete, read again ..."

The console printing information on normal and error is as follows (due to the fact that there are too many lines of print, I use "..." instead of a partial duplicate line):

1Connected to the target VM, address: ' 127.0.0.1:62194 ', Transport: ' Socket '2Log4j:warn No appenders could be found forLogger (io.netty.util.internal.logging.InternalLoggerFactory).3 Log4j:warn Initialize the log4j system properly.4 Read data: this time 1024; cumulative 1024; Total 5748425 Read data: this time 1024; cumulative 2048; Total 5748426 Read data: this time 1024; cumulative 3072; Total 5748427 ......8 Read data: this time 1024; cumulative 572416; Total 5748429 Read data: this time 1024; cumulative 573440; Total 574842Ten Read data: this time 1024; cumulative 574464; Total 574842 One Read data: this time 378; cumulative 574842; Total 574842 A0 ~ ~NULL -Disconnected from the target VM, address: ' 127.0.0.1:62194 ', Transport: ' Socket ' -  theProcess finished with exit code 0

Normal Results

1Connected to the target VM, address: ' 127.0.0.1:62068 ', Transport: ' Socket '2Log4j:warn No appenders could be found forLogger (io.netty.util.internal.logging.InternalLoggerFactory).3 Log4j:warn Initialize the log4j system properly.4 Read data: this time 1024; cumulative 1024; Total 5748425 Read data: this time 1024; cumulative 2048; Total 5748426 Read data: this time 1024; cumulative 3072; Total 5748427 Read data: this time 1024; cumulative 4096; Total 5748428 Read data: this time 1024; cumulative 5120; Total 5748429 Read data: this time 1024; cumulative 6144; Total 574842Ten Read data: this time 1024; cumulative 7168; Total 574842 One Read data: this time 1024; cumulative 8192; Total 574842 A Read data: this time 1024; cumulative 9216; Total 574842 - Read data: this time 1024; cumulative 10240; Total 574842 - Read data: this time 1024; cumulative 11264; Total 574842 the Read data: this time 1024; cumulative 12288; Total 574842 - Read data: this time 1024; cumulative 13312; Total 574842 - Read data: this time 576; cumulative 13888; Total 574842 -******read data incomplete, read again ... + Read data: this time 16384; cumulative 30272; Total 574842 - Read data: this time 16384; cumulative 46656; Total 574842 + Read data: this time 16384; cumulative 63040; Total 574842 A ...... at Read data: this time 16384; cumulative 554560; Total 574842 - Read data: this time 16384; cumulative 570944; Total 574842 - Read data: this time 3898; cumulative 574842; Total 574842 -0 ~ ~NULL -Disconnected from the target VM, address: ' 127.0.0.1:62068 ', Transport: ' Socket ' -  inProcess finished with exit code 0

result when error occurs

Results Last print 0~~null indicates normal end (Returns a code of 0 error message null).

Report:

Issue before processing console print results

1 "C:\Program Files ... 2  for Logger (io.netty.util.internal.logging.InternalLoggerFactory). 3 Log4j:warn Initialize the log4j system properly. 4 -1 ~ ~ service exception; Detail:java.lang.NullPointerException56 Process finished with exit Code 0

issue before processing console printing information

Since the Channelread () method was not executed, the data I obtained was not able to perform the assignment operation and reported a null pointer exception.

results Last Printed-1 ~ ~ Service exception; Detail:java.lang.NullPointerException indicates an exception occurred (Return code 1; Error message "Service exception; Detail:java.lang.NullPointerException") .

A read incomplete bug fix record that occurs when a large data volume message is received using the netty4.x client