Learn Java NIO design through the Http11NioProtocol source code of Tomcat

Learn Java NIO design through the Http11NioProtocol source code of Tomcat

Tomcat's Http11NioProtocol uses Java NIO technology to implement high-performance Web servers. This article analyzes the source code of Http11NioProtocol to learn how to use Java NIO. We can learn about the combination of blocking IO and non-blocking IO, the read and write operations of NIO, and the use of Selector. wakeup.

1. Initialization phase
The first step of Java NIO server implementation is to open a new ServerSocketChannel object. The implementation of Http11NioProtocol is no exception. You can see this code in the init method of the NioEndPoint class.

Code 1: NioEndPoint. init () method

Public void init ()
Throws Exception {

If (initialized)
Return;
// Start a new ServerSocketChannel
ServerSock = ServerSocketChannel. open ();
// Set the performance preference of the socket
ServerSock. socket (). setPerformancePreferences (socketProperties. getPerformanceConnectionTime (),
SocketProperties. getPerformanceLatency (),
SocketProperties. getPerformanceBandwidth ());
InetSocketAddress addr = (address! = Null? New InetSocketAddress (address, port): new InetSocketAddress (port ));
// Bind the port number and set the backlog
ServerSock. socket (). bind (addr, backlog );
// Set serverSock to block IO
ServerSock. configureBlocking (true); // mimic APR behavior

// Number of acceptor threads initialized
If (acceptorThreadCount = 0 ){
// FIXME: Doesn't seem to work that well with multiple accept threads
AcceptorThreadCount = 1;
}
// Initialize the number of poller threads
If (pollerThreadCount <= 0 ){
// Minimum one poller thread
PollerThreadCount = 1;
}

// Initialize SSL as needed
// This code is omitted because it mainly focuses on Java NIO.
If (isSSLEnabled ()){
......
}
// OutOfMemoryError Policy
If (oomParachute> 0) reclaimParachute (true );

// Enable NioSelectorPool
SelectorPool. open ();
Initialized = true;
}

 

In the NioEndPoint. init method, we can see that the ServerSocketChannel is set to block IO and no ready events are registered. In this way, you can use the blocking accept method as easily as blocking ServerSocket to receive new connections from the client. However, when the NioEndPoint. Accept thread obtains a new SocketChannel through the accept method, it constructs an OP_REGISTER-type PollerEvent event and stores it in the Poller. events queue. When we use ServerSocket to implement the server, after receiving a new connection, we usually retrieve a thread from the thread pool to process the connection.

In the setSocketOptions method of NioEndPoint. Accept, you can see the processing process after obtaining the SocketChannel. The procedure is as follows:

1) set SocketChannel to non-blocking;

2) construct a PollerEvent object of the OP_REGISTER type and put it in the Poller. events queue.

Code 2: setSocketOptions method of the NioEndPoint. Accept class

Protected boolean setSocketOptions (SocketChannel socket ){
Try {
// Set the client Socket to non-blocking, APR Style
Socket. configureBlocking (false );
Socket sock = socket. socket ();
SocketProperties. setProperties (sock );
// Obtain the NioChannel object from the cache. If not, build
NioChannel channel = nioChannels. poll ();
If (channel = null ){
// If sslContext is not equal to null, you need to start ssl
If (sslContext! = Null ){
....
}
// Normal tcp start
Else {
// Construct a NioBufferHandler object
NioBufferHandler bufhandler = new NioBufferHandler (socketProperties. getAppReadBufSize (),
SocketProperties. getAppWriteBufSize (),
SocketProperties. getDirectBuffer ());
// Construct a NioChannel object
Channel = new NioChannel (socket, bufhandler );
}
} Else {
// Obtain the NioChannel object from the cache and set the client socket
Channel. setIOChannel (socket );
If (channel instanceof SecureNioChannel ){
SSLEngine engine = createSSLEngine ();
(SecureNioChannel) channel). reset (engine );
} Else {
Channel. reset ();
}
}
// Register a NioChannel object
GetPoller0 (). register (channel );
} Catch (Throwable t ){
Try {
Log. error ("", t );
} Catch (Throwable tt ){}
// Tell to close the socket
Return false;
}
Return true;
}

The Poller thread extracts the PollerEvent object from the Poller. events queue and runs the PollerEvent. run () method. If the OP_REGISTER event is found in the PollerEvent. run () method, the OP_READ readiness event of the SocketChannel object will be registered on Poller. selector.

Code 3: PollerEvent. run () code snippet

Public void run (){
If (interestOps = OP_REGISTER ){
Try {
// Register the OP_READ readiness event on Poller. selector
Socket. getIOChannel (). register (socket. getPoller (). getSelector (), SelectionKey. OP_READ, key );
} Catch (Exception x ){
Log. error ("", x );
}
}
......
}

Now, the client connection preparation is complete. We obtained a SocketChannel from the client and registered the OP_READ readiness event to Poller. selector (1 ). Now you can read and write data.

Figure 1: initialization status of ServerSocketChannel and SocketChannel

2. the wakeup method of Poller. selector
The Poller thread will do the following:

1) obtain the number of selected selectionkeys through the selection operation;

2) execute PollerEvent in the Poller. events queue;

3) process the selected SelectionKey.

When a new PollerEvent object is added to the Poller. events queue, You need to perform Step 2 as soon as possible without blocking the selection operation. Therefore, we need to use the Selector. wakeup () method to achieve this requirement. Tomcat uses the semaphore wakeupCounter to control the Selector. wakeup () method, blocking the use of the Selector. select () method and the non-blocking Selector. selectNow () method.

When a new PollerEvent object is added to the Poller. events queue, the Selector. wakeup () method is executed as follows.

When the value of wakeupCounter is equal to 0 after 1, it indicates that Poller. selector is blocked in the selection operation. In this case, the Selector. wakeup () method must be called.
If the value of 1 is not equal to 0 after the value of wakeupCounter, it indicates that Poller. selector is not blocked in the selection operation. Therefore, you do not need to call the Selector. wakeup () method. To execute Step 2 as soon as possible, the Poller thread directly calls the non-blocking method Selector. selectNow () next time ().
Code 4: Poller. addEvent () method to add the PollerEvent object to the Poller. events queue.

Public void addEvent (Runnable event ){
Events. offer (event );
// Call the wakeup method if the value of wakeupCount is 0 after 1.
If (wakeupCounter. incrementAndGet () = 0) selector. wakeup ();
}

Code 5: selection operation code of the poroller thread
If (wakeupCounter. get ()> 0 ){
KeyCount = selector. selectNow ();
Else {
WakeupCounter. set (-1 );
KeyCount = selector. select (selectorTimeout );
}
WakeupCounter. set (0 );

This design has the following features:
Calling the wakeup () method on the Selector object will cause the first unreturned selection operation to return immediately. If no selection operation is performed, the next call to the select () method will be immediately returned. This will delay the wakeup behavior to the next select () method is often not what we want (of course not what Tomcat wants ). We generally only want to wake up from the sleeping thread, but allow the next selection operation to process normally.
Therefore, Tomcat controls the change of the wakeupCounter semaphore to call the Selector. wakeup () method only when the selection operation is blocked. When a new PollerEvent object is added to the Poller. events queue and is not blocked in the selection operation, the non-blocking method Selector. selectNow () is called directly ().

For more details, please continue to read the highlights on the next page:

1 2 Next Page