In-depth Netty source parsing one of the data structures

Netty is an asynchronous event-driven network application framework that is suitable for fast development and maintenance of high-performance protocols for both server and client. Its schema is as follows:

Its core is divided into three parts,

The lowest layer is a custom byte buffer that supports 0 copy functions;

The middle tier is a generic communication API;

The upper layer is an extensible event model.

Now we start with a custom byte buffer that supports 0 copy functionality from the lowest layer, which is contained within the Io.netty.buffer package.

Io.netty.buffer Package Description :

The Io.netty.buffer package contains the netty underlying data structure.

In Java NiO bytebuffer the underlying data structures that represent the underlying binary and textual information, Io.netty.buffer abstracted Bytebuffer api,netty using its own buffer API to provide bytebuffer of NIO to represent a sequence of bytes. Its buffer API has significant advantages over the use of Bytebuffer. Netty buffer type: BYTEBUF's design fundamentally solves bytebuffer problems and satisfies the daily needs of Web application developers. Here are some of the more cool features:

You can define your own buffer type according to your own needs.

Transparent 0 copies are implemented using the built-in composite buffer type.

Like StringBuffer, support for dynamic buffer types expands buffer capacity as needed.

You no longer need to call the flip () method.

The situation is usually faster than bytebuffer.

Improved scalability

The BYTEBUF provides a rich set of operations for optimizing fast protocol implementations. For example, it provides several ways to get the unsigned value and the string and the retrieval of a specific byte sequence in buffer, and you can add more convenient access by extending or wrapping the existing buffer type. The custom buffer type is still inherited from the Bytebuf interface instead of introducing an incompatible new type.

Transparent 0 copies

To maximize the performance of your network applications, you need to reduce the number of memory copy operations. Of course you can also set up a set of buffer that can be sliced and use them to assemble a complete message. Netty provides a combination of buffer, which allows you to create a new buffer from any number of existing bufer without using a memory copy. For example, a message consists of two parts: the head and the content. In a modular application, when sending a message, the two parts can have different module generation and subsequent combinations.

+--------+----------+
| header | Body |
+--------+----------+

If you use Bytebuffer (Java NIO), you must create a new large buffer and then copy the header and contents to the newly created buffer, or you can use the write operation in NIO to set the operation. But if you use compound buffer as a bytebuffer array instead of just one buffer, the abstract type is broken and a complex state management is introduced. And if you don't read or write from the NIO channel, it won't work.

Combination types and component types do not match

New Bytebuffer[] {header, body};

Instead, BYTEBUF does not have this problem because of its high extensibility and built-in combo buffer type.

// combination types and component types do not match Bytebuf message = Unpooled.wrappedbuffer (header, body); // Therefore, you can create a combo buffer by mixing a combo buffer and a normal buffer  = unpooled.wrappedbuffer (message, footer); // since the combo buffer is still a bytebuf, you can easily get its content, even if the area you want to get across multiple components, and get the simple buffer to get the same way.   The unsigned integral type obtained in the//instance spans the content and tail. -Footer.readablebytes ()-1); 

Capacity auto-expansion (Automatic capacity Extension)

Many protocols define the length of a message, which means that the length of the message cannot be determined before the message is created or the length of the message is not easily calculated. Just like you just started creating a string. We usually estimate the length of the string and then use StringBuffer to expand as needed.

// creates a new dynamic buffer. Internally, in order to avoid potential waste of memory space, the real buffer will be deferred for creation. = Unpooled.buffer (4);  // a buffer of size 4 is created internally when the first attempt is made to write b.writebyte (' 1 '); B.writebyte (' 2 '); B.writebyte (' 3 '); B.writebyte (' 4 ') ; // when the number of bytes to be written exceeds the initialized capacity 4 o'clock, in-house, buffer automatically re-allocates a larger capacity b.writebyte (' 5 ');

Better performance

In most cases, the buffer that inherits from Bytebuf is very lightweight in the wrapper for byte arrays (for example, byte[]). Unlike Bytebuffer,bytebuf, where there is no complex boundary check and index compensation, it is easier for the JVM to optimize the way that buffer is obtained.

The more complex buffer implementations are used only for slicing or combining buffer, and the complex buffer has the same performance as Bytebuffer.

The inheritance relationship of Bytebuf

Enter BYTEBUF to see:

BYTE provides a random or sequential acquisition of a sequence of bytes that can read 0 or more bytes.

This interface provides an abstract attempt on one or more basic byte arrays (byte[]) and ordinary NiO bytebuffer.

Create a buffer

It is recommended that you create a new buffer by using the helper method unpooled instead of calling a constructor that implements the buffer.

random access to an index

Like a normal byte array, BYTEBUF uses the 0-based index method, which means that the first byte of the byte array is indexed to 0, and the last byte of the array is the capacity-1. For example, to facilitate all the bytes of a buffer without considering its internal implementation, you can do this:

Buffer = ...;    for (int i = 0; I < buffer.capacity (); i + +)      {byte b = buffer.getbyte (i);      System.out.println ((char) b);  }

Sequential acquisition of indexes

BYTEBUF provides two pointers to support sequential reads and writes: Readerindex () is used for read operations, and Writerindex () is used for write operations. Shows how a buffer is divided into 3 regions by 2 pointers:

       +-------------------+------------------+------------------+       | can discard bytes  |  Readable bytes |  Can write section  |       |            |  (content)   |          |       +-------------------+-------------- +------------------+       |            |           |          |       0      <=  Read index   <=  Write index <=   capacity

Readable bytes (real content)

This part is the area where the data is actually stored, and all operations that begin with a read or skip will read or skip the data at the current read index and increment according to the number of bytes read. If the parameter of the read operation is also a bytebuf and does not indicate the destination index, the write index of the specified buffer will be incremented synchronously.

If there is no content below (then the read will report an out-of-bounds exception), the new assigned default value of buffer or the readable index of the copied buffer is 0.

Iterate through a buffer of readable bytes

Buffer = ...;  while (Buffer.readable ()) {   System.out.println (Buffer.readbyte ());}

Writable bytes

This area is an undefined space that needs to be populated. Any operation ending with write writes data at the current writable index and increases the writable index based on the number of bytes written. If the parameter of the write operation is Bytebuf, and the source index is not specified, the specified buffer's readable index synchronization increases.

The write index of the default value of buffer is the capacity of buffer if there is no writable content (which continues to report an out-of-bounds exception).

  // fills the writable area of the buffer with any integral type.  {@link Bytebuf} buffer = ...;    while (Buffer.maxwritablebytes () >= 4) {      buffer.writeint (Random.nextint ());  }

Bytes that can be discarded

This area contains bytes that have been read by the read operation. The capacity of the zone is 0 when initialized, but its capacity gradually reaches the write index when the read operation is performed. Declare the no zone by calling the Discardreadbytes () method, as shown in the description:

discardreadbytes () method before: *      +-------------------+------------------+------------------+ *      | discardable bytes |  Readable bytes  |  Writable bytes  | *      +-------------------+------------------+------------------+ *      |                  |                  | | *      0      <=      readerindex   <=   writerindex    <=    * *      +------------------+--------------------------------------+ *      |  Readable bytes  |    Writable bytes (got more space)   | *      +------------------+--------------------------------------+ *      |                  |                                      | * Readerindex (0) <= writerindex (decreased)        <=        capacity

Note: After calling the Discardreadbytes () method, it is not possible to guarantee the contents of a byte. The write-in section does not move in most cases, and it can even populate completely different data based on different buffer implementations.

Clear the buffer index

You can set the value of Readerindex () and Writerindex () to 0.clear () by calling the clear () method without clearing the contents of buffer and simply setting the value of two pointers to 0. Note: Bytebuf Clear () The syntax of the method and the Bytebuffer clear () operation are completely different.

* * *      +-------------------+------------------+------------------+ *      | discardable bytes |  Readable bytes  |  Writable bytes  | *      +-------------------+------------------+------------------+ *      |                  |                  | | *      0      <=      readerindex   <=   writerindex    <=    ** * * *      +---------------------------------------------------------+ *      |             Writable bytes (got more space)             | *      +---------------------------------------------------------+ *      |                                                         | *      0 = Readerindex = Writerindex            <=            capacity

Retrieve Operation:

For simple single-byte retrieval, use IndexOf (), Bytesbefore (). Bytesbefore () is particularly useful when dealing with null (the trailing character).

For complex searches, use Foreachbyte ().

Label and Reset

Each buffer has two index tags. One is used to store readerindex, and the other is used to store Writerindex (). You can also reset the position of the two indexes by calling the Reset method.

In addition to the InputStream label and reset method without Readlimit, it also works.

SOURCE Buffer

You can create a view that already exists in buffer by calling the duplicate () or slice method. Source buffer has independent readerindex, Writeindex, and label indexes, but shares some of the other internal data, like NiO buffer.

Call the Copy () method when you need to fully copy a buffer that already exists.

Convert to an existing JDK type

byte array

Determines whether a buffer consists of a byte array, using the HasArray () method;

If a buffer is composed of a byte array, it can be obtained directly from the array () method;

NIO Buffer

Determine if a buffer can be converted to NiO's buffer, using Niobuffercount () to determine

If a bytebuf can be converted into NiO bytebuffer, it can be obtained by Niobuffer method.

String

There are a number of ToString methods that convert Bytebuf to string, so be sure to note that ToString is not a conversion method.

I/O flow

Please refer to Bytebufinputstream and Bytebufoutputstream.

Summary:

Netty the bottom of the data structure for the BYTEBUF interface and its implementation, seize them on the bottom of the implementation of the essence, this article is only for the bytebuf to do a brief introduction, the implementation of the class also needs the reader to slowly grope.

In-depth Netty source parsing one of the data structures