Netty Source analysis of "Java NiO's previous incarnation of this present" __java

Brief Introduction

Java NIO is an asynchronous IO introduced by Java 1.4.
Java NIO consists of the following core components:
-Channel
-Buffer
- comparison of Selector NIO and IO

The difference between IO and NiO is mainly reflected in three aspects: IO based on stream (stream oriented), while NiO is blocked based on buffer (buffer oriented) IO operations, and NIO operations are non-blocking IO without selector Concept, while NIO has a selector concept. Stream based and Buffer based

Traditional IO is oriented to byte stream or character streams, whereas in NIO we discard the traditional IO stream and introduce the concept of Channel and Buffer . In NIO, I can only read data from Channel to buffer or write data from buffer to Channel.
So what is based on flow ? In a typical Java IO operation, we read one or more bytes sequentially from one stream in a streaming fashion, so we can't arbitrarily change the position of the reading pointer.
and based on the Buffer it seems a little different. We first need to read the data from the Channel to the buffer, when there is data in the buffer, we can operate on the data. Unlike IO, which is a sequential operation, we are free to read data from anywhere in NIO. blocking and non-blocking

Java provides a variety of Stream operations that are blocked, such as when we call a read method to read the contents of a file, then the thread that calls read is blocked until the read operation completes.
The non-blocking mode of NIO allows us to perform IO operations in a non-blocking manner. For example, we need to read data from the network, in the non-blocking mode of NIO, when we call the Read method, read reads and returns if there is data at this time; If there is no data at this time, read returns directly without blocking the current thread. selector

Selector is a concept in NiO that is the key to Java NiO's ability to perform IO operations in a non-blocking manner.
By Selector, a thread can listen to multiple Channel IO events, and when we register Channel with a Selector, the mechanism within Selector can automatically query (select) These registered Channe L have a ready IO event (for example, readable, writable, network connection complete, etc.). With such a Selector mechanism, we can easily manage multiple Channel with a single thread. Java NIO Channel

Typically, all NIO I/O operations start with Channel. A channel is similar to a stream.
Java stream and NIO Channel contrast we can perform read and write operations in the same Channel, while the same Stream only supports reading or writing. Channel can read and write asynchronously, and Stream is a blocking synchronous read-write. Channel always reads data from the buffer or writes the data to the buffer.

Channel types are: FileChannel, file Operation Datagramchannel, UDP operation Socketchannel, TCP operation Serversocketchannel, TCP operation, use on server side.

These channels cover both UDP and TCP network IO and file IO.
Basic examples of Channel use:

public static void Main (string[] args) throws Exception
{
    randomaccessfile afile = new Randomaccessfile ("/users/ Xiongyongshun/settings.xml "," RW ");
    FileChannel Inchannel = Afile.getchannel ();

    Bytebuffer buf = Bytebuffer.allocate (in);

    int bytesread = Inchannel.read (BUF);
    while (Bytesread!=-1) {
        buf.flip ();

        while (Buf.hasremaining ()) {
            System.out.print ((char) buf.get ());
        }

        Buf.clear ();
        Bytesread = Inchannel.read (BUF);
    }
    Afile.close ();
}

FileChannel

FileChannel is the channel of manipulating files, we can read data from a file FileChannel, or write data to a file.
Note that FileChannel cannot be set to non-blocking mode. Open FileChannel

Randomaccessfile afile     = new Randomaccessfile ("Data/nio-data.txt", "RW");
FileChannel      Inchannel = Afile.getchannel ();

reading data from FileChannel

Bytebuffer buf = Bytebuffer.allocate (in);
int bytesread = Inchannel.read (BUF);

Write Data

String NewData = "New String to write to file ..." + System.currenttimemillis ();

Bytebuffer buf = Bytebuffer.allocate (in);
Buf.clear ();
Buf.put (Newdata.getbytes ());

Buf.flip ();

while (Buf.hasremaining ()) {
    channel.write (BUF);
}

off

When we complete the operation on FileChannel, we must close it

Set Position

Long pos channel.position ();
Channel.position (pos +123);

File Size

We can get the size of the file associated with this channel by Channel.size (). Note that this returns the size of the file, not the number of elements remaining in the Channel. truncate file

Channel.truncate (1024);

Truncates the size of the file to 1024 bytes. Force Write

We can force the writing of the cached unread data to the file:

Channel.force (TRUE);

Socketchannel

Socketchannel is a Channel that a client uses to make a TCP connection.
There are two ways to create a socketchannel: Open a socketchannel and connect it to a server when a Serversocketchannel accepts a connection request, it returns a Socketchannel object. Open Socketchannel

Socketchannel Socketchannel = Socketchannel.open ();
Socketchannel.connect (New Inetsocketaddress ("http://example.com", 80));

off

reading Data

Bytebuffer buf = Bytebuffer.allocate (in);
int bytesread = Socketchannel.read (BUF);

If read () returns -1, then the connection is interrupted. Write Data

String NewData = "New String to write to file ..." + System.currenttimemillis ();

Bytebuffer buf = Bytebuffer.allocate (in);
Buf.clear ();
Buf.put (Newdata.getbytes ());

Buf.flip ();

while (Buf.hasremaining ()) {
    channel.write (BUF);
}

non-blocking mode

We can set Socketchannel to asynchronous mode so that our connect, read, and write are asynchronous. Connection

Socketchannel.configureblocking (false);
Socketchannel.connect (New Inetsocketaddress ("http://example.com");

while (! Socketchannel.finishconnect ()) {
    //wait, or do something else ...    
}

In asynchronous mode, perhaps the connection has not yet been established, and the Connect method is returned, so we need to check whether the current is connected to the host, so that it is judged by a while loop. Read and write

In asynchronous mode, the way to read and write is the same.
When reading, because it is asynchronous, we must check the return value of read to determine whether the data is currently being read. Serversocketchannel

Serversocketchannel, as the name suggests, is used on the server side, can listen to the client's TCP connection, for example:

Serversocketchannel Serversocketchannel = Serversocketchannel.open ();
Serversocketchannel.socket (). bind (New Inetsocketaddress (9999));
while (true) {
    Socketchannel Socketchannel =
            serversocketchannel.accept ();

    Do something with Socketchannel ...
}

Turn off

Serversocketchannel Serversocketchannel = Serversocketchannel.open ();

Serversocketchannel.close ();

Listening for connections

We can use the Serversocketchannel.accept () method to listen for TCP connection requests from the client, and the Accept () method blocks until a connection arrives, and when there is a connection, this method returns a Socketchannel object:

while (true) {
    Socketchannel Socketchannel =
            serversocketchannel.accept ();

    Do something with Socketchannel ...
}

non-blocking mode

In non-blocking mode, the Accept () is non-blocking, so if no connection arrives at this point, the Accept () method returns null:

Serversocketchannel Serversocketchannel = Serversocketchannel.open ();

Serversocketchannel.socket (). bind (New Inetsocketaddress (9999));
Serversocketchannel.configureblocking (false);

while (true) {
    Socketchannel Socketchannel =
            serversocketchannel.accept ();

    if (Socketchannel!= null) {
        //do something with Socketchannel ...
        }
}

Datagramchannel

The Datagramchannel is used to handle UDP connections. Open

Datagramchannel channel = Datagramchannel.open ();
Channel.socket (). bind (New Inetsocketaddress (9999));

reading Data

Bytebuffer buf = Bytebuffer.allocate (in);
Buf.clear ();

Channel.receive (BUF);

Send Data

String NewData = "New String to write to file ..."
                    + system.currenttimemillis ();

Bytebuffer buf = Bytebuffer.allocate (in);
Buf.clear ();
Buf.put (Newdata.getbytes ());
Buf.flip ();

int bytessent = Channel.send (buf, New inetsocketaddress ("example.com", 80));

connect to a specified address

Because UDP is connectionless, this connect is not a true connection to TCP, but rather it speaks datagramchannel locks, so we can read or write data only from the specified address.

Channel.connect (New Inetsocketaddress ("example.com", 80));

Java NIO Buffer

When we need to interact with the NIO Channel, we need to use the NIO buffer, which reads the data from the buffer to the Channel, and writes it to the buffer from the Channel.
In fact, a Buffer is actually an area of memory where we can read and write data in this memory area. NIO Buffer is actually a encapsulation of such a block of memory, and provides some operational methods that allow us to easily read and write data.
Buffer types are: Bytebuffer charbuffer doublebuffer floatbuffer intbuffer longbuffer

These Buffer covers all the Java basic data types that can be transmitted from IO. Basic use of NIO Buffer

The steps for using NIO buffer are to write data into buffer. Invokes the Buffer.flip () method to convert the NIO buffer to read mode. Read data from buffer to call the Buffer.clear () or Buffer.compact () method to convert the buffer to write mode.

When we write the data into the buffer, buffer records how much data we have written, and when we need to read the data from the buffer, we must call Buffer.flip () to switch the buffer to read mode.
Once all the buffer data has been read, we have to clear the buffer so that it can be written anew, and the buffer may be called buffer.clear () or buffer.compact ().
For example:

public class Test {public
    static void Main (string[] args) {
        Intbuffer intbuffer = intbuffer.allocate (2);
        Intbuffer.put (12345678);
        Intbuffer.put (2);
        Intbuffer.flip ();
        System.err.println (Intbuffer.get ());
        System.err.println (Intbuffer.get ());
    }

In this case, we assign two units of Intbuffer, so it can write two int values.
We use the Put method to write the int value, then use the Flip method to convert the buffer to read mode, and then use the Get method to obtain the two int values from the buffer continuously.
Whenever a Get method is invoked to read data, the read pointer of the buffer moves forward one unit length (here is an int length) buffer Property

A Buffer has three attributes: capacity position limit

The meaning of position and limit is related to buffer in read mode or write mode, and the meaning of capacity is independent of the mode of buffer. Capacity

A block of memory will have a fixed size, that is, the capacity (capacity), we write up to capacity units of data into the Buffer, such as a doublebuffer, its capacity is 100, then we can write 100 Double data. Position

When writing data from a buffer, we begin writing from a definite location (position) of the buffer. In the initial state, the position value is 0. Every time we write a unit of data, position increments one.
When we read the data from the Buffer, we also read it from a specific location. when we call the Filp () method to convert the Buffer from write mode to read mode, the position value is automatically set to 0, and every time we read one unit of data, the position value is incremented by 1.
position represents the position pointer of a read-write operation. Limit

Limit-position indicates how many units of data can be written/read at this time.
For example, in write mode, if limit is at this time, position is 2, it means that 2 units of data have been written, and 10-2 = 8 units of data can be written. Example:

public class Test {public
    static void Main (String args[]) {
        Intbuffer intbuffer = intbuffer.allocate;
        Intbuffer.put (ten);
        Intbuffer.put ();
        SYSTEM.ERR.PRINTLN ("Write mode:");
        System.err.println ("\tcapacity:" + intbuffer.capacity ());
        System.err.println ("\tposition:" + intbuffer.position ());
        System.err.println ("\tlimit:" + intbuffer.limit ());

        Intbuffer.flip ();
        SYSTEM.ERR.PRINTLN ("Read mode:");
        System.err.println ("\tcapacity:" + intbuffer.capacity ());
        System.err.println ("\tposition:" + intbuffer.position ());
        System.err.println ("\tlimit:" + intbuffer.limit ());
    }

Here we first write two int values, at this time capacity = ten, Position = 2, limit = 10.
Then we call flip to read mode, at which point capacity = 0, limit = 2; Allocate Buffer

In order to get a Buffer object, we first need to allocate the memory space. Each type of buffer has a allocate () method that we can use to allocate buffer:

Bytebuffer buf = Bytebuffer.allocate (48);

Here we have allocated the memory space of sizeof (byte) bytes.

Charbuffer buf = charbuffer.allocate (1024);

Here we assign a size of 1024 characters of buffer, that is, this buffer can store 1024 Char, its size is 1024 * 2 bytes. on the difference between Direct buffer and Non-direct buffer

Direct Buffer: The allocated memory is not on the JVM heap and is not managed by GC. (However, the Java object of direct buffer is managed by GC, so that when the GC occurs, the direct buffer is also released) because direct buffer is not allocated on the JVM heap, so direct buffe The impact of R on the memory footprint of the application is less obvious (it actually takes up so much memory, but the JVM is not good at counting the memory that is not managed by the JVM.) The overhead of applying and releasing Direct Buffer is relatively high. So the correct way to use Direct buffer is to apply for a buffer during initialization, and then reuse this buffer to release the buffer at the end of the program. When using Direct Buffer, it is more efficient to perform some underlying system IO operations because the JVM does not need to copy memory from the buffer into the intermediate temporary buffer at this time.

non-direct Buffer: the allocation of memory directly on the JVM heap, essentially the encapsulation of a byte[] array. Because the non-direct buffer is in the JVM heap, the memory of this buffer is copied into the intermediate temporary buffer when operating in the underlying IO operation of the OS. Therefore, the efficiency of Non-direct Buffer is lower. Write data to Buffer

int bytesread = Inchannel.read (BUF); Read into buffer.
Buf.put (127);

reading data from Buffer

Read from buffer into channel.
int byteswritten = Inchannel.write (BUF);
byte Abyte = Buf.get ();

Reset Position

The Buffer.rewind () method resets the position value to 0, so we can read/write the Buffer again.
In the case of read mode, the reset is the position of the read mode, and if it is write mode, the position of the write mode is reset.
For example:

public class Test {public
    static void Main (string[] args) {
        Intbuffer intbuffer = intbuffer.allocate (2);
        Intbuffer.put (1);
        Intbuffer.put (2);
        System.err.println ("Position:" + intbuffer.position ());

        Intbuffer.rewind ();
        System.err.println ("Position:" + intbuffer.position ());
        Intbuffer.put (1);
        Intbuffer.put (2);
        System.err.println ("Position:" + intbuffer.position ());


        Intbuffer.flip ();
        System.err.println ("Position:" + intbuffer.position ());
        Intbuffer.get ();
        Intbuffer.get ();
        System.err.println ("Position:" + intbuffer.position ());

        Intbuffer.rewind ();
        System.err.println ("Position:" + intbuffer.position ());
    }

Rewind () is mainly for read mode. When reading to limit, you can call the Rewind () method and set the read position to 0. about Mark () and reset ()

We can save the current position value by calling Buffer.mark (), and then return the position value by calling the Buffer.reset () method.
For example:

public class Test {public
    static void Main (string[] args) {
        Intbuffer intbuffer = intbuffer.allocate (2);
        Intbuffer.put (1);
        Intbuffer.put (2);
        Intbuffer.flip ();
        System.err.println (Intbuffer.get ());
        System.err.println ("Position:" + intbuffer.position ());
        Intbuffer.mark ();
        System.err.println (Intbuffer.get ());

        System.err.println ("Position:" + intbuffer.position ());
        Intbuffer.reset ();
        System.err.println ("Position:" + intbuffer.position ());
        System.err.println (Intbuffer.get ());
    }

Here we write two int values and then read a value first. The value of the read position at this time is 1.
Then we call the mark () method to save the current position (in read mode, so the read position is saved), and then read again, and position is 2 now.
Then use Reset () to restore the original read position, so read the position is 1, you can read the data again. the difference between flip, rewind and clear Flip

Method source code:

Public final Buffer Flip () {
    limit = position;
    Position = 0;
    Mark =-1;
    return this;
}

The read/write mode of Buffer is shared with a position and limit variable.
When you change from write mode to read mode, the original writing position becomes the limitof the reading mode. Rewind

Method source Code

Public final Buffer Rewind () {
    position = 0;
    Mark =-1;
    return this;
}

Rewind, that is, rewind, this method is simply to place the position to 0. Clear

Method source code:

Public final Buffer Clear () {
    position = 0;
    Limit = capacity;
    Mark =-1;
    return this;
}

As we can tell from the source, clear sets Positin to 0 and sets limit to capacity.
The clear method uses the scene:
-In a buffer that has already been filled with data, call clear to read the buffer data from scratch.
-To fill a buffer with data, you can call clear and write until you reach limit. Example:

Intbuffer Intbuffer = intbuffer.allocate (2);
Intbuffer.flip ();
System.err.println ("Position:" + intbuffer.position ());
System.err.println ("Limit:" + intbuffer.limit ());
SYSTEM.ERR.PRINTLN ("Capacity:" + intbuffer.capacity ());

Cannot read here, because limit = = Position = 0, no data.
System.err.println (Intbuffer.get ());

Intbuffer.clear ();
System.err.println ("Position:" + intbuffer.position ());
System.err.println ("Limit:" + intbuffer.limit ());
SYSTEM.ERR.PRINTLN ("Capacity:" + intbuffer.capacity ());

Here you can read the data because clear, limit = = Capacity = 2, Position = = 0,
//Even if we do not write any data into the buffer.
System.err.println (Intbuffer.get ()); Read to 0
System.err.println (intbuffer.get ());//Read to 0

Comparison of Buffer

We can compare two buffer by equals () or CompareTo () method, and when and only if the following conditions are met, two buffer is equal: Two buffer is the same type of two buffer and the number of remaining data is the same two Buffe The rest of the data for R is the same.

Through the above conditions, we can find that compared with two buffer, not every element in buffer is compared, but the remaining elements in the buffer. Selector

Selector allows a single thread to manipulate multiple Channel. If more than one Channel is used in our application, it is convenient to use Selector to do so, but because multiple Channel are used in one thread, the efficiency of each Channel transmission is reduced.
The diagram using Selector is as follows:

In order to use Selector, we first need to register Channel in Selector, then call the Selector Select () method, which blocks until registered in Selector