Chapter 5 of Netty In Action: Buffers and nettyinaction
Note: This article is from Netty In Action;
NOTE: If your original translation is reprinted, please indicate the source!
This chapter introduces
- ByteBuf
- ByteBufHolder
- ByteBufAllocator
- Use these interfaces to allocate buffering and perform operations
Whenever you need to transmit data, it must contain a buffer. The buffer classes provided by Java NIO APIs are quite limited and have not been optimized. Using JDK's ByteBuffer operations is more complex. Buffer is an important component of APIs. Netty provides a powerful buffer implementation to represent a byte sequence and helps you operate on the original byte or custom POJO. Netty's ByteBuf is equivalent to JDK's ByteBuffer. ByteBuf is used to transmit data through channels in Netty. It was redesigned to solve some problems in JDK's ByteBuffer, making it more efficient for developers to develop network applications. This chapter describes the buffer zone in Netty and explains why it is better than the built-in buffer implementation in JDK. It also describes how to use ByteBuf in Netty to access data and how to use it.
5.1 Buffer API
Netty's buffer API has two interfaces:
When Netty uses reference-counting (reference count), it knows that Buf and other resources are safely released. Although it knows that Netty uses reference count effectively, this is automatically completed. This allows Netty to use pools and other techniques to speed up and keep the memory usage at a normal level. You don't need to do anything to achieve this, but when developing a Netty application, you should process the data and release the pool resources as soon as possible.
The Netty buffer API provides several advantages:
- You can customize the buffer type.
- Implement zero copy through a built-in compound buffer type
- Good scalability, such as StringBuffer
- You do not need to call flip () to switch the read/write mode.
- Separate read and write Indexes
- Method chain
- Reference count
- Pooling (Pool)
5.2 ByteBuf-byte data container
To interact remotely, you must send/receive data in bytecode. For various reasons, an efficient, convenient, and easy-to-use data interface is required, and Netty's ByteBuf meets these needs. ByteBuf is a well-optimized data container, we can effectively add byte data to ByteBuf or retrieve data from ByteBuf. ByteBuf has two parts: one for reading and the other for writing. We can read data in order and jump to start to read it again. For all data operations, we only need to adjust the data index and start the read operation again. 5.2.1 how does ByteBuf work?
After data is written to ByteBuf, the number of bytes added when the index is written. After reading bytes, the read index is increased. You can read the byte until the data is written to the same location as the read index. If you continue reading the data for a number of times, IndexOutOfBoundsException will be thrown. Any method that calls ByteBuf to start reading/writing will independently maintain the Read and Write indexes. The default maximum capacity limit of ByteBuf is Integer. MAX_VALUE. If the value is exceeded, an exception occurs. ByteBuf is similar to a byte array. The biggest difference is that the Read and Write indexes can be used to control access to the buffer data. A ByteBuf with a capacity of 16 is displayed:
5.2.2 when using Netty for different types of ByteBuf, three different types of ByteBuf will be encountered
Heap Buffer (Heap Buffer)The most common type is that ByteBuf stores data in the JVM heap space, which is achieved by storing data in arrays. The heap buffer can be quickly allocated and released when not in use. It also provides a method to directly access the array and obtain byte [] data through ByteBuf. array. Access to the ByteBuf array in the non-heap buffer zone will cause UnsupportedOperationException. You can use ByteBuf. hasArray () to check whether the access to the array is supported.
Direct Buffer (Direct Buffer)Direct buffer, directly allocating memory outside the heap. The direct buffer does not occupy the heap space capacity. You should consider the maximum memory capacity used by the application and how to limit it. The performance of the direct buffer zone is good when data is transmitted using Socket, because if an indirect buffer zone is used, JVM will first copy the data to the direct buffer zone and then transmit it; however, the disadvantage of direct buffer is that it is more complex than the heap buffer when allocating memory space and releasing memory. Netty uses the memory pool to solve this problem, this is one of the reasons Netty uses the memory pool. Direct buffer does not support array access to data, but we can indirectly access the data array, as shown in the following code:
ByteBuf directBuf = Unpooled.directBuffer(16);if(!directBuf.hasArray()){int len = directBuf.readableBytes();byte[] arr = new byte[len];directBuf.getBytes(0, arr);}Accessing the data array in the direct buffer zone requires more encoding and more complex operations. It is recommended that you use the heap buffer to access data in the array. Composite Buffer (Composite Buffer) Composite Buffer, we can create multiple different ByteBuf, and then provide a view of these ByteBuf combinations. The Composite Buffer is like a list. We can dynamically add and delete the ByteBuf. JDK's ByteBuffer does not have this function. Netty provides the CompositeByteBuf class to process the compound buffer. CompositeByteBuf is only a view, and CompositeByteBuf. hasArray () always returns false because it may contain different types of bytes Buf, either directly or indirectly. For example, a message consists of two parts: header and body. The header and body are assembled into a message and sent. The body may be the same, but the header is different, with CompositeByteBuf, you do not need to allocate a new buffer every time. Show the CompositeByteBuf header and body:
If you use JDK's ByteBuffer, you can only create an array or a new ByteBuffer, and then copy the content to the new ByteBuffer. The following is an example of using CompositeByteBuf:
CompositeByteBuf compBuf = Unpooled. compositeBuffer (); ByteBuf heapBuf = Unpooled. buffer (8); ByteBuf directBuf = Unpooled. directBuffer (16); // Add ByteBuf to CompositeByteBufcompBuf. addComponents (heapBuf, directBuf); // Delete the first ByteBufcompBuf. removeComponent (0); Iterator <ByteBuf> iter = compBuf. iterator (); while (iter. hasNext () {System. out. println (iter. next (). toString ();} // use an array to access data if (! CompBuf. hasArray () {int len = compBuf. readableBytes (); byte [] arr = new byte [len]; compBuf. getBytes (0, arr );}CompositeByteBuf is a subclass of ByteBuf. We can operate CompositeByteBuf like BytBuf. In addition, Netty optimizes socket read/write operations by using CompositeByteBuf as much as possible. Using CompositeByteBuf will not cause memory leakage. 5.3 ByteBuf byte operations
ByteBuf provides many operations that allow you to modify the data content or only read data. ByteBuf is similar to JDK's ByteBuffer, but ByteBuf provides better performance. 5.3.1 Random Access Index
ByteBuf uses zero-based-indexing (index starting from 0). The index of the first byte is 0, and the index of the last byte is the capacity-1 of ByteBuf, the following code traverses all ByteBuf Bytes:
//create a ByteBuf of capacity is 16ByteBuf buf = Unpooled.buffer(16);//write data to buffor(int i=0;i<16;i++){buf.writeByte(i+1);}//read data from buffor(int i=0;i<buf.capacity();i++){System.out.println(buf.getByte(i));}Note that reading and writing indexes will not be promoted during index access. We can use ByteBuf's readerIndex () or writerIndex () to promote reading or writing indexes separately. 5.3.2 sequential access index
ByteBuf provides two pointer variables to pay for read and write operations. Read operations use readerIndex () and write operations use writerIndex (). This is different from JDK's ByteBuffer. ByteBuffer has only one method to set indexes. Therefore, you need to use the flip () method to switch between read and write modes. ByteBuf must comply with: 0 <= readerIndex <= writerIndex <= capacity.
5.3.3 Discardable bytes discard bytes
We can call ByteBuf. discardReadBytes () to retrieve the bytes that have been read. discardReadBytes () will discard the bytes from index 0 to readerIndex. After the discardReadBytes () method is called, it is changed:
ByteBuf. discardReadBytes () can be used to clear the data read from ByteBuf, so that ByteBuf has extra space to accommodate new data. However, discardReadBytes () may involve memory replication, because it needs to move the readable bytes in ByteBuf to the starting position, this operation will affect the performance. Generally, the use of memory will be more profitable when the memory needs to be released immediately. 5.3.4 readable bytes (actual content)
ReaderIndex will be added for any read operation. If the read operation parameter is also a ByteBuf and the target index is not specified, the writerIndex of the specified destination buffer will be added together. If there is not enough content, IndexOutOfBoundException will be thrown. The default value of readerIndex for the newly allocated, packaged, and replicated buffers is 0. The following code obtains all readable data:
ByteBuf buf = Unpooled.buffer(16);while(buf.isReadable()){System.out.println(buf.readByte());}(The code is different from the original book. The original book may be based on versions earlier than Netty4, which is based on Netty4) 5.3.5 Writable bytes
WriterIndex is added for any write operation. If the write operation parameter is also a ByteBuf and the data source index is not specified, the readerIndex of the specified buffer will also be added. If there are not enough writable bytes, IndexOutOfBoundException will be thrown. The default value of the newly allocated buffer writerIndex is 0. The following code shows a random int number to fill the buffer until the buffer space is exhausted:
Random random = new Random();ByteBuf buf = Unpooled.buffer(16);while(buf.writableBytes() >= 4){buf.writeInt(random.nextInt());}5.3.6 clear the buffer Index Clearing the buffer indexs
You can call ByteBuf. clear () to set readerIndex and writerIndex to 0. clear () does not clear the buffer, but sets the two index values to 0. Note that ByteBuf. clear () and JDK's ByteBuffer. clear () have different meanings. It indicates that before ByteBuf calls clear:
After clear () is called:
Clear () is cheaper than discardReadBytes () Because clear () does not replicate any memory. 5.3.7 Search operations
Various indexOf () methods help you determine whether the index of a value is consistent. We can use ByteBufProcessor to perform a complex and dynamic sequential search for a simple static single-byte search. If you want to decode variable-length data, such as a string ending with null, you will find that the bytesBefore (byte value) method is useful. For example, we write an integrated flash sockets application, which uses the NULL end content and the bytesBefore (byte value) method to easily check the NULL bytes in the data. If we do not have ByteBufProcessor, we need to do these tasks by ourselves and use ByteBufProcessor for better efficiency. 5.3.8 standard and reset Mark and reset
Each ByteBuf has two labeled indexes, one storing readerIndex and the other storing writerIndex. You can call a reset method to reposition one of the two indexes. It is similar to the annotation and reset method of InputStream and has no read restrictions. We can call readerIndex (int readerIndex) and writerIndex (int writerIndex) to move the read index and write index to the specified location. When we call these two methods to set the specified index location, IndexOutOfBoundException may be thrown. 5.3.9 derivative buffer zone Derived buffers
Call duplicate (), slice (), slice (int index, int length), order (ByteOrder endianness) to create a view of the existing buffer. The derived buffer zone has independent readerIndex, writerIndex, and annotation index. If you need a new copy of the existing buffer, you can use copy () or copy (int index, int length. See the following code:
// get a Charset of UTF-8Charset utf8 = Charset.forName("UTF-8");// get a ByteBufByteBuf buf = Unpooled.copiedBuffer("Netty in Action rocks!", utf8);// sliceByteBuf sliced = buf.slice(0, 14);// copyByteBuf copy = buf.copy(0, 14);// print "Netty in Action rocks!"System.out.println(buf.toString(utf8));// print "Netty in Act"System.out.println(sliced.toString(utf8));// print "Netty in Act"System.out.println(copy.toString(utf8));5.3.10 read/write operations and other operations
There are two main types of read/write operations:
- The get/set operation is based on the index and sets or obtains bytes in the given index.
- Reads and writes from the current index, increasing the current write index or read Index
For various read/write methods or other Check Methods of ByteBuf, refer to the source code of ByteBuf. I will not go into details here. 5.4 ByteBufHolder
ByteBufHolder is a helper class and an interface. Its implementation class is DefaultByteBufHolder, and some other interface classes that implement the ByteBufHolder interface. The function of ByteBufHolder is to help you more easily access data in ByteBuf. When the buffer zone is useless, you can use this helper class to release resources. ByteBufHolder is simple and provides few methods for access. If you want to implement a "message object" payload stored in ByteBuf, it is a good idea to use ByteBufHolder. Although the various buffer implementation classes provided by Netty are easy to use, Netty still provides some tool classes for ease of creating and using various buffers. The following describes some buffer tool classes in Netty. 5.4.1 ByteBufAllocator
Netty supports various ByteBuf pool implementations to make it possible for Netty to provide a type called ByteBufAllocator. ByteBufAllocator is responsible for allocating ByteBuf instances. ByteBufAllocator provides various methods for allocating different ByteBuf instances. If you need a heap buffer, you can use ByteBufAllocator. heapBuffer (), which requires a direct buffer. You can use ByteBufAllocator. directBuffer (). You can use ByteBufAllocator to create a composite buffer. compositeBuffer (). For other methods, see ByteBufAllocator source code and comments. It is easy to obtain the ByteBufAllocator object, which can be obtained from alloc () of the Channel or alloc () of ChannelHandlerContext. See the following code:
ServerBootstrap b = new ServerBootstrap();b.group(group).channel(NioServerSocketChannel.class).localAddress(new InetSocketAddress(port)).childHandler(new ChannelInitializer<SocketChannel>() {@Overrideprotected void initChannel(SocketChannel ch) throws Exception {// get ByteBufAllocator instance by Channel.alloc()ByteBufAllocator alloc0 = ch.alloc();ch.pipeline().addLast(new ChannelInboundHandlerAdapter() {@Overridepublic void channelActive(ChannelHandlerContext ctx) throws Exception {//get ByteBufAllocator instance by ChannelHandlerContext.alloc()ByteBufAllocator alloc1 = ctx.alloc();ctx.writeAndFlush(buf.duplicate()).addListener(ChannelFutureListener.CLOSE);}});}});Netty has two different ByteBufAllocator implementations. One implementation of the ByteBuf instance pool minimizes the allocation and recovery costs and memory usage, and the other is to create a new ByteBuf instance each time it is used. Netty uses PooledByteBufAllocator by default. We can change it through ChannelConfig or by setting a different implementation through guidance. More details are described later.
5.4.2 Unpooled
Unpooled is also a tool class used to create a buffer. It is easy to use Unpooled. Unpooled provides many methods. For detailed methods and usage, see the API documentation or Netty source code. See the following code:
// Create a CompositeByteBuf compBuf = Unpooled. compositeBuffer (); // create a heap buffer ByteBuf heapBuf = Unpooled. buffer (8); // create a direct buffer ByteBuf directBuf = Unpooled. directBuffer (16 );
5.4.3 ByteBufUtil
ByteBufUtil provides some static methods, which are useful when operating ByteBuf. ByteBufUtil provides methods other than Unpooled. Perhaps the most valuable method is the hexDump (ByteBuf buffer) method. This method returns the hexadecimal string of the readable bytes in the specified ByteBuf, it can be used to print ByteBuf content during program debugging. The hexadecimal string is more user-friendly than the byte string. 5.5 Summary
This chapter describes how to create and use the buffer class ByteBuf provided by Netty and some tool classes for operating the ByteBuf.
Register for the computer examination of Anhui Provincial personnel department title
[Top] outline of the Sichuan Computer Rank Examination Level 1 examination
1. Basic computer knowledge: The development and characteristics of electronic computers, the classification and application of electronic computers
2. Representation of information in the computer: The concept of carry counting; representation of binary, octal, decimal, and hexadecimal
For numbers and characters,
Internal code, external code); Computer Information Representation Unit: Bit, byte, word; Storage
Capacity concept
3. Composition of computer systems: Composition of computer hardware systems, classification and functions of each part; computer software systems;
Relationship between software and hardware in computer systems; computer work processes; computer's
Main Performance Indicators (Word Length, internal and external memory capacity, computing speed, etc)
4. Basic knowledge of computer networks: definition, function, and classification of computer networks; basic knowledge and application of Intrernet (
Email, file transmission, WWW, etc.; basic IP address and Domain Name System
Concept)
5. Basic multimedia technology knowledge: Basic Concepts and main application fields of Multimedia Technology
6. Computer Virus knowledge: definition and prevention of computer viruses
Chapter 2 DOS Operating System
Summary of this chapter:
1. Basic Components of Dos and boot
2. File concept, file naming rules, and file directories (tree structure, path, current directory, and absolute directory of directories)
Path and relative path)
3. Common Dos Commands: DIR, CD, MD, COPY, DEL, REN, FORMAT, TYPE, XCOPY,
DELTREE
4. Functions and settings of the system configuration file CONFIG. SYS (FILES, BUFFERS, DEVICE)
5. batch processing concepts and automatic batch processing files
Chapter 3 Windows Operating System
Key content of this chapter:
1. Features of Windows 95
2. Composition and operations of the Windows 95 graphical user interface (desktop, window, dialog box, icon, Start Menu, and task)
Column)
3. management functions of Windows 95:
① Functions and operations of "My Computer" and "Resource Manager"
② Browsing computer resources
③ Create, copy, search, move, delete, and rename files and folders
④ Disk management
4. Windows 95 online help
Chapter 4 Chinese Text Processing System Word97
Summary of this chapter:
1. Basic operations on a document (create, open, close, and save)
2. Basic operations for text editing (move the cursor; insert, rewrite, delete, move, and copy characters and text blocks;
String search and replacement)
3. Document Layout (format adjustment of pages, paragraphs, and characters)
4. Use of styles and templates
5. Tables and Images
Chapter 5 Chinese workbook Excel97
Summary of this chapter:
1. Basic concepts of workbooks
2. Create and edit a worksheet
3. Use formulas and functions (Auto SUM, SUM, AVERAGE, COUNT, MAX, MIN)
4. Concepts of workbooks and operations between multiple worksheets
5. Create charts
6. Data Processing (sorting, filtering, and summary of Records)
Appendix: requirements for level 1 exams
1. master basic computer knowledge
2. Understand the basic components of computer systems and their working processes
3. Understand the functions of the microcomputer operating system and have the basic ability to use the microcomputer operating system.
4. master various Chinese Character Input Methods, master the basic knowledge of word processing, and have the ability to use a word processing software.
5. master the basic knowledge of workbooks and have the basic ability to use workbooks.
6. Understand basic computer network knowledge
7. Master Computer Security usage knowledge... the remaining full text>
