Introduction and use of Netty buffer buffer

   Whenever you need to transfer data, it must contain a buffer. The Java NIO API has a very limited number of buffer classes and is not optimized, and bytebuffer operations using JDK are more complex. A buffer is an important build, and it is part of the API. Netty provides a powerful buffer implementation to represent a sequence of bytes and helps you manipulate the original byte or custom POJO. The Netty bytebuf equivalent to the bytebuffer,bytebuf of JDK is the transmission of data through Channel in Netty. It has been redesigned to address some of the problems in the JDK's bytebuffer, making it more efficient for developers to develop Web applications.

the Netty buffer API has two interfaces

One: Bytebuf
When you need to interact with the remote, you need to send/receive data in bytecode. For a variety of reasons, an efficient, convenient, easy-to-use data interface is necessary, and Netty Bytebuf to meet these needs, BYTEBUF is a good optimized data container, we can effectively add byte data to Bytebuf or from the BYTEBUF to obtain data.Bytebuf has 2 parts: one for reading and one for writing。 We can read the data sequentially, and we can skip to the beginning and reread it. All the data we need to do is adjust the index of reading data and start reading again.
After writing data to Bytebuf, the write index is the number of bytes added. Reading the index increases after the bytes have been read. You can read bytes until the index is written to the same location as the index, and Indexoutofboundsexception is thrown if the number continues to read. Any method that calls Bytebuf to start read/write maintains both read and write indexes separately. The default maximum capacity limit for bytebuf is integer.max_value, which will cause an exception if the value is exceeded.
Bytebuf is similar to a byte array, and the biggest difference is that read and write indexes can be used to control access to buffer data.
3 different types of BYTEBUF are encountered when using Netty: Heap buffer (heap buffers)
The most commonly used type is that BYTEBUF stores data in the JVM's heap space by storing the data in an array implementation. Heap buffers can be quickly allocated and can be quickly released when not in use. It also provides a way to access the array directly by Bytebuf.array () to get the byte[data. Accessing an array of non-heap buffer bytebuf causes unsupportedoperationexception, and you can use Bytebuf.hasarray () to check whether the array is supported.Direct buffer (immediate buffers)
A direct buffer that allocates memory directly beyond the heap. The direct buffer does not consume heap space capacity, and should be used with the maximum memory capacity to be used by the application and how to limit it. The direct buffer performs well when passing data using the Socket, because using an indirect buffer, the JVM replicates the data to the direct buffer before it is passed, but the disadvantage of the direct buffer is that it is more complex than the heap buffer when allocating memory space and freeing memory, and Netty uses the memory pool to solve the problem. This is one of the reasons why Netty uses memory pools. The direct buffer does not support array access data, but we can indirectly access the array of data, such as the following code:

    Bytebuf directbuf =  unpooled.directbuffer ();
    if (!directbuf.hasarray ()) {
       int len =  directbuf.readablebytes ();
       byte[] arr = new Byte[len];
       Directbuf.getbytes (0, arr);
    

composite buffer (composite buffers)
Composite buffers, we can create multiple different bytebuf and then provide one of these bytebuf
A grouped view. Composite buffers are like a list, and we can dynamically add and remove BYTEBUF,JDK Bytebuffer without such functionality. Netty provides a compositebytebuf class to handle a composite buffer, compositebytebuf is just a view, and Compositebytebuf.hasarray () always returns false because it May contain some direct or indirect bytebuf of different types. The following are examples of using compositebytebuf:

Compositebytebuf compbuf =unpooled.compositebuffer ();
 Bytebuf heapbuf = Unpooled.buffer (8);
 Bytebuf directbuf = Unpooled.directbuffer (16);//Add bytebuf to
 compositebytebuf compbuf.addcomponents  
 (heapBuf , directbuf);//delete the first bytebuf    compbuf.removecomponent (0);
 Iterator<bytebuf> iter = Compbuf.iterator ();
 while (Iter.hasnext ()) {
   System.out.println (Iter.next ()). ToString ())  
 ;
   Use array Access data
   if (!compbuf.hasarray ()) {
      int len = Compbuf.readablebytes ();
      byte[] arr = new Byte[len];
      Compbuf.getbytes (0, arr);
   

Compositebytebuf is a subclass of Bytebuf, we can manipulate compositebytebuf as we do bytbuf. and Netty optimize socket read and write operation is to use COMPOSITEBYTEBUF to do as much as possible, using COMPOSITEBYTEBUF does not operate memory leak problem.

Bytebuf byte operation API: Random Access Index
BYTEBUF uses zero-based-indexing (index from 0), the index of the first byte is 0, and the last byte index is Bytebuf capacity-1, and the following code is all the bytes that traverse the Bytebuf:

Create   a bytebuf of capacity is 
   bytebuf buf = Unpooled.buffer (b);
   Write data to BUF for
   (int i=0;i<16;i++) {
         buf.writebyte (i+1); 
   } 
   Read data from BUF for
   (int i=0;i<buf.capacity (); i++) {
        System.out.println (buf.getbyte (i)); 
  }

Note that reading indexes and writing indexes are not pushed through index access, and we can pass the BYTEBUF
Readerindex () or writerindex () to advance the read index or write index separately.

Sequential Access Index
BYTEBUF provides two pointer variables to pay read and write operations, read operation is using Readerindex (), write operation using Writerindex (). Unlike JDK Bytebuffer, Bytebuffer has only one way to set the index, so you need to use the flip () method to switch between read and write mode. Bytebuf must meet:
0 <= readerindex <= writerindex <=capacity.

Discardable bytes Discarded bytes
We can call Bytebuf.discardreadbytes () to reclaim bytes that have been read, discardreadbytes () will discard the bytes from index 0 to Readerindex.
Bytebuf.discardreadbytes () can be used to empty the read data in the BYTEBUF so that Bytebuf has extra space to accommodate the new data, but discardreadbytes () may involve memory replication because it needs to move Bytebuf can be read bytes to the start, such operations can affect performance, generally in the need to release the memory immediately when the use of revenue will be relatively large. Readable bytes (actual content)
Any read operation increases Readerindex if the parameter of the read operation is also a bytebuf
Instead of specifying a destination index, the writerindex of the specified destination buffer is incremented together, and Indexoutofboundexception is thrown when there is not enough content. The default value for the Readerindex of newly allocated, wrapped, and replicated buffers is 0. The following code shows the acquisition of all readable data:

   Bytebuf buf = Unpooled.buffer ();
   while (Buf.isreadable ()) {
      System.out.println (Buf.readbyte ());
   }

Writable bytes
Writable bytes Any write operation increases the Writerindex. If the parameter of the write operation is also a bytebuf and the data source index is not specified, the readerindex of the specified buffer also increases together. If there are not enough writable bytes, the indexoutofboundexception will be thrown. The default value for the newly allocated buffer writerindex is 0. The following code shows a random int number to fill the buffer until the buffer space is exhausted:

      Random Random = new Random ();
      Bytebuf buf = Unpooled.buffer ();
      while (Buf.writablebytes () >= 4) {
         buf.writeint (Random.nextint ());
      }

Clear Buffer Index
Clearing the buffer Indexs call Bytebuf.clear () can set Readerindex and Writerindex to 0,clear () do not purge the contents of the buffers, just set two index values to 0. Note that the semantics of bytebuf.clear () are different from JDK's Bytebuffer.clear (). Clear () is cheap compared to discardreadbytes () because clear () does not replicate any memory. Search operations
Search operations various indexOf () methods help you locate the index of a value that matches, we can use Bytebufprocessor Complex dynamic sequential search realizes a simple static single byte search. If you want to decode variable-length data, such as a null-terminated string, you will find that the Bytesbefore (byte value) method is useful. For example, we write an integrated Flash sockets application that uses null-terminated content, using the Bytesbefore (byte value) method to easily check for empty bytes in the data. Without bytebufprocessor, we need to do these things ourselves and use bytebufprocessor more efficiently. The standard and reset Mark and reset each bytebuf has two annotation indices, one storage readerindex, one storage writerindex. You can reposition one of the two indexes by calling a reset method, similar to the InputStream annotation and reset method, with no read restrictions. We can move the read and write indexes to the specified location by calling readerindex (int readerindex) and writerindex (int writerindex). Calling the two method settings at the specified index position may throw a indexoutofboundexception. The derived buffer derived buffers calls duplicate (), slice (), slice (int index, int length), order (Byteorder endianness) Creates a view of an existing buffer. The derived buffer has a separate readerindex, Writerindex, and callout index. If you need a fresh copy of an existing buffer, you can use copy () or copy (int index, int length) to get it.

Get a Charset of UTF-8
 Charset UTF8 = charset.forname ("UTF-8");
 Get a bytebuf
 bytebuf buf = Unpooled.copiedbuffer ("Netty in Action rocks!", UTF8);
 Slice  
 bytebuf sliced = buf.slice (0); 
 Copy
 bytebuf copy = buf.copy (0);
 Print "" Netty in Action   rocks! "" 
 System.out.println (buf.tostring (UTF8));
Print "Netty in  Act"
System.out.println (sliced.tostring (UTF8));
Print "Netty in Act"
System.out.println (copy.tostring (UTF8));

Read/write operations and other operations have two main types of read and write operations: The get/set operation is indexed and reads and writes at the given index set or gets bytes from the current index, incrementing the current write index or read index

Two: Bytebufholder

Bytebufholder is an auxiliary class, an interface whose implementation class is Defaultbytebufholder and some other interface classes that implement the Bytebufholder interface. The role of Bytebufholder is to help access the data in Bytebuf more easily, and you can use this helper class to free up resources when the buffer is useless. Bytebufholder is simple and offers few accessible methods. If you want to implement a "message object" payload stored in Bytebuf, using Bytebufholder is a good idea.

Although the various buffer implementation classes provided by Netty are already easy to use, Netty still provides a number of tool classes to use, making it easier to create and use a variety of buffers. The buffer tool classes in some Netty are described below. The Bytebufallocator Netty supports a variety of BYTEBUF pool implementations to enable Netty to provide a possibility called bytebufallocator. Bytebufallocator is responsible for allocating BYTEBUF instances, Bytebufallocator provides various methods for assigning different bytebuf, such as the need for a heap buffer to use Bytebufallocator.heapbuffer (), A direct buffer is required to use Bytebufallocator.directbuffer () and a composite buffer is required to use Bytebufallocator.compositebuffer (). The use of other methods can be seen Bytebufallocator source code and comments. It is easy to get the Bytebufallocator object, either from the Channel alloc () or from the Channelhandlercontext alloc (). Look at the following code:

Serverbootstrap B = new Serverbootstrap ();
 New Inetsocketaddress (port)). Childhandler (
    new channelinitializer<socketchannel> () {
  @Override
  protected void Initchannel (Socketchannel ch) throws Exception {
  //Get  Bytebufallocator instance by Channel.alloc ()
  bytebufallocator alloc0 = Ch.alloc ();
  Ch.pipeline (). AddLast (New Channelinboundhandleradapter () {
   @Override public
   void Channelactive ( Channelhandlercontext ctx)
   throws exception{
   //get bytebufallocator by Channelhandlercontext.alloc ()
   Ctx.writeandflush (Buf.duplicate ()). AddListener (
       channelfuturelistener.close);}});

Netty has two different bytebufallocator implementations, one implementation BYTEBUF instance pooling minimizes allocation and recovery costs and memory usage, and another implementation is to create a new BYTEBUF instance each time. Netty use Pooledbytebufallocator By default, we can change it by channelconfig or by booting a different implementation. More details are described later. Unpooled
Unpooled is also a tool class used to create buffers, and unpooled is also easy to use. Unpooled provides a number of methods, detailed methods and use to see API documentation or Netty source code. Look at the following code:

  Create a composite buffer 
  compositebytebuf compbuf = Unpooled.compositebuffer ();
  Create heap buffer
  bytebuf heapbuf = Unpooled.buffer (8);
  Create a direct buffer
  bytebuf directbuf = Unpooled.directbuffer (16);

Bytebufutil
Bytebufutil provides a number of static methods that are useful when manipulating bytebuf. Bytebufutil provides some methods beyond unpooled, perhaps the most valuable is the Hexdump (bytebuf buffer) method, which returns a hexadecimal string of readable bytes in the specified bytebuf, which can be printed when debugging a program Bytebuf , hexadecimal strings are friendlier to users than bytes. The Netty buffer API offers several advantages

You can customize the buffer type

0 copies implemented with a built-in composite buffer type

Good scalability, like StringBuffer.

You do not need to invoke flip () to toggle read/write mode

Read and write index separate

Method Chain

Reference count

Pooling (pool)