Netty in Action (22) Chapter 11th Part II Netty provides some native handler and codecs

11.4 decoding delimited and length-based protocols

If you use Netty to work, you will often encounter the use of delimiters or specify the number of bytes to decode the method, the next section will explain the Netty in dealing with this problem to give some of the specific implementation

11.4.1 Delimited Protocols

The delimiter protocol uses a fixed character to mark a message or the beginning and end of an information end, a segmented message block we call a "frame", and there are many formal specification agreements defined by the RFC, such as Smtp,pop3,imap,telnet, of course, Many private organizations often have their own proprietary format, no matter which protocol you use, and some of the decoders listed in Table 11.5 will help you customize some of the decoders to help you extract frame data for any separator cut.

Figure 11.5 shows how some of the data in the frame data is segmented with \ r \ n.

The following code listing shows how to use Linebasedframedecoder to implement the problem shown in Figure 11.5

If the frame data you are using is segmented by a delimiter, rather than splitting it at the end of the behavior, you can use Delimiterbasedframedecoder to simplify the processing of the frame data in this scenario, simply by specifying a specific separator in the constructor.

These decoders are your tools for implementing a custom decoder, and we can use the following scenario descriptions as an example:

1) The input data stream contains multiple data frames, with each data frame ending with \ n

2) Each data frame contains multiple items, each of which is segmented by a space character

3) Each frame represents a command that has a command name and multiple parameters.

We give the following classes to the custom decoder clusters given in the above protocol:

1) CMD--------Store the contents of a frame or a command in a bytebuf, including the name and parameters that store the command

2) Cmddecoder-----------to retrieve a row of data by overriding the Decode method, constructing one or more instances of CMD from this row of data

3) Cmdhandler------------get the Cmd object from the Cmddecoder and do some work on the object

4) Cmdhandlerinitializer----------------for convenience, we embed some of the previously defined classes into the handler chain of the pipeline through specialized channelinitializer.

The complete code listing, as shown in 11.9, is the core of the decoder that implements the Linebasedframedecoder

11.4.2 length-based Protocols

A length-based specification defines the length of a frame data at the head of this frame data, rather than using some specified characters to slice the characters, table 11.6 shows the 2 decoder provided by Netty to handle this type of specification protocol

Figure 11.6 shows how a fixed frame-size Fixedlengthframedecoder decoder handles 8-byte frame data.

You often encounter a scene, the size of the frame data is not a fixed, in order to deal with the size of this data frame is a variable problem we can use Lengthfieldbasedframedecoder, this class can be from each frame data header information know the size of the frame data, Then get the corresponding size frame data from the data stream

Figure 11.7 shows an example in which the length of the frame data header takes up 2 bytes, the subscript starts at 0, and the length has 2 bytes

Lengthfieldbasedframedecoder provides multiple constructors to cover the changing scene of frame data head data, Listing 11.10 uses a constructor, the three entry parameters of this constructor are Maxframelength,lengthfieldoffset and Lengthfieldlength, in which the length of each frame is defined in the first eight bytes of the frame

So far, you have learned several decoders provided by Netty, these decoders are used to support the structure of a variety of data frames, the structure of these data frames include the following, one is a certain character segmentation and formation of the data frame, but the fixed-length data frame, three is a variable length of the data frame, The length of the data frame is specified at the head of the frame, and as more protocols and specifications are formed you will find and write more and more various decoders and classify and document these decoders to form a more complete system.

11.5 Writing Big Data

How to write large chunks of data efficiently in an asynchronous framework can become a very special problem because of the possibility of network saturation, because the write operations are not blocked, they even return to completion state when all the data is not fully written and notify Channelfuture when this happens. You're going to risk a memory overflow and the write operation cannot be terminated, so when you write a large chunk of data, you need to be prepared to handle a case where a slow connection to a remote end produces a delay in releasing memory, and we take an example to write a file to the network

In our previous 4.2 chapters discussed in the transport protocol we mentioned the characteristics of NiO 0 copies, this feature will eliminate the copy process, when the file system needs to move a file in the network, these processes will occur in the Netty core module, so in this practice, the application needs to give the interface fileregion a A specific implementation, this interface is defined in the Netty documentation: the region of a file needs to be sent through the channel to support 0 copy transfer

The following code listing shows how you can transfer a file content using 0 copy technology, create a defaultfileregion from FileInputStream, and write it to the channel

This example can only apply the direct file content transfer, the system can not have any action on the file, if there is the operation of the file you need to copy the data from the file system to the user memory, you can use Chunkedwritehandler, It supports the asynchronous writing of large data streams without high memory loss

The core of this function is interface Chunkedinput<b>, where B refers to the type returned by the method Readchunk method, this interface gives four implementations, As shown in table 11.7 below, each represents a data stream without a defined length consumed by the Chunkedwritehandler

Code Listing 11.12 illustrates the use of Chunkedstream, a type of implementation that is often used in real-world scenarios, which also shows the initialization of files and Sslcontext, which initializes the channel with a chain processor when the Initchannel method is called.

When the channel's state is activated, Writestreamhandler will write the file block as a chunkedstream and encrypt it using Sslhandler before the file is transferred.

Block input: Use your own Chunkedinput implementation to install a Chunkedwritehandler in the pipeline

In this section, we explain how to use the 0 copy feature for efficient transmission, how to transfer big data without risking oom------using Chunkedwritehandler, and in the next section we'll cover several ways to serialize POJOs

11.6 Serializing Data

The JDK provides ObjectOutputStream and objectinputstream for serializing and deserializing native data types and Pojo for network transmission, and this series of APIs is not very complex, can be applied to any object that implements the Java.io.Serializable interface, but this is not a very efficient method, and in this section we will look at some of the tool classes that Netty provides for serialization

11.6.1 JDK Serialization

If your app uses ObjectOutputStream and ObjectInputStream to interact with the remote side, the problem with compatibility is your main concern, and the serialization of the JDK will be the best way to solve this problem, Table 11.8 shows an introduction to some of the JDK-based serialization classes provided by Netty

11.6.2 serialization with JBoss marshalling

If you are free to choose to use some other dependencies, then JBoss's marshalling will be a good choice, it will be 3 times times faster than the JDK serialization and the serialized content appears more compact

Netty provides support for JBoss marshalling, in the form of decoders and encoders, As shown in table 11.9, the first block is compatible with the implementation of the JDK serialization, and the second is to ensure compatibility while providing maximum performance, which is implemented using JBoss marshalling

The following code listing shows you how to use Marshallingdecoder and Marshallingencoder, which requires almost the correct configuration channelpipeline.

、

11.6.3 serialization via Protocol buffers

Netty the last solution to serialization is to use a decoder protocol buffers, the PROTOBUF is developed by Google users to interact with the data, is now open source, the source code can be https://github.com/ Found on the GOOGLE/PROTOBUF website

PROTOBUF's structured data is encoded and decoded in a more compact and efficient way, and it has now been implemented in multiple languages, enabling it to be better applied to cross-language projects. Table 11.10 shows the implementation of the Channelhandler provided by Netty for PROTOBUF support

Here, once again, we just need to add the appropriate channelhandler to Channelpipeline, as shown in listing 11.14 below:

In this section, we describe several serialization methods provided by Netty, which are given in the form of encoding and decoding, and these serialization methods are based on the following specific implementations of the 1) JDK serialization method, JBoss marshalling and Google's Protobuf

11.7 Summary

Several decoders and processors provided by Netty native can be used in conjunction to handle a wide variety of business scenarios, and these components have proven robust in the use of many large business projects

Please note that during our introduction to this section, we are only using the most commonly used APIs, and more detailed and sophisticated APIs are available in the API documentation

In the next section, we will learn about another protocol that is more advanced to improve the performance of Web development Websocket,netty WebSocket provides a lot of tools that you can use to make it easy to work with WebSocket

Netty in Action (22) Chapter 11th Part II Netty provides some native handler and codecs