Comparison of two high-performance I/O design modes (reactor/proactor)

Reprinted: http://www.cppblog.com/pansunyou/archive/2011/01/26/io_design_patterns.html

System I/O can be classified into blocking type, non-blocking synchronous type, and non-blocking asynchronous type [1,

2]. blocking I/O means that the control will be returned to the caller only after the call operation is complete. as a result, the caller is blocked and cannot do anything else during this time. even more depressing is that the thread where the caller is located cannot free up his/her hands to respond to other requests while waiting for the IO results. This is a waste of resources. Take the read () operation for example. The code that calls this function will be stuck here until the data in the socket cache it reads arrives.

In contrast, non-blocking synchronization immediately returns control to the caller. The caller does not need to wait. The caller obtains two results from the called function: either the call is successful, or the system returns an error ID to indicate that the caller's current resource is unavailable, wait or try again. For example, in the read () operation, if the current socket has no data readable, the system returns immediatelyEwoulblock/eagainTo the read () caller, "data is not ready yet. Try again later ".

Non-blocking asynchronous calls are slightly different. When the function is returned immediately, the caller is also notified that the request has started. The system will use another resource or thread to complete the call operation, and notify the caller (for example, through the callback function) when the call is completed ). Take windowsReadfile ()Or POSIXAio_read ()After it is called, the function returns immediately, and the operating system starts read operations at the same time in the background.

Among the above three types of Io, non-blocking Asynchronization provides the highest performance and the best scalability.

This article explores different I/O utilization mechanisms and provides a cross-platform design mode (solution). We hope this article can help TCP high-performance server developers and select the best design scheme. Next, we will compare the implementation and performance of the discussed solutions in Java, C #, and C ++. we will not mention the blocking scheme later in this article, because the blocking I/O is really not scalable, and the performance cannot meet the requirements of high-performance servers.

Two Io multiplexing solutions: reactor and proactor

In general, the I/O reuse mechanism is requiredEvent sharer(Event demultiplexor [1,

3]). the role of the event sharer is to distribute the read/write event sources to the handlers of various read/write events. It is like a courier shouting downstairs: Who has sent anything? Come and get it. At the beginning, developers need to register events of interest in the sharer and provide corresponding event handlers or callback functions; when appropriate, the event sharer will distribute the requested events to these handler or callback functions.

The two modes that involve the event sharer are called reactor and proactor [1]. the reactor mode is based on synchronous I/O, while the proactor mode is related to asynchronous I/O. in reactor mode, the event separator waits for an event or an application or operation to occur (for example, the file descriptor can be read/written, or the socket can be read/written ), the event splitter transmits the event to the previously registered event handler or callback function, and the latter performs actual read/write operations.

In proactor mode, the event handler (or initiated on behalf of the event separator) directly initiates an asynchronous read/write operation (equivalent to a request), and the actual work is done by the operating system. When initiating a request, you must provide the following parameters: the cache used to store read data, the size of read data, or the cache used to store outgoing data, and the callback function after the request is completed. The event splitter knows the request, and silently waits for the request to be completed, and then forwards the event to the corresponding event handler or callback. For example, on Windows, the event handler ships an asynchronous Io operation (with overlapped technology), and The iocompletion event such as the event separator completes [1].
The typical implementation of this asynchronous mode is based on the underlying asynchronous API of the operating system. Therefore, we can call it "system-level" or "true" Asynchronous, because the specific read/write operations are performed by the operating system.

Another example is to better understand the differences between the reactor and proactor modes. Here we only focus on the read operation, because the write operation is similar. The reactor practices are as follows:

• An event handler claims that it is very interested in reading events on a socket;
• The event separator is waiting for the occurrence of the event;
• When an event occurs, the event splitter is awakened, which notifies the previous event handler;
• The event handler receives the message and reads data from the socket. If necessary, it claims to be interested in the read event on the socket and repeats the preceding steps;

Next let's take a look at how the real asynchronous mode proactor works:

• The event handler ships a write operation directly (of course, the operating system must support this asynchronous operation ). at this time, the event handler does not care about the read event at all. It only sends such a request, and it is dreaming of thisWrite operation completion event. This processor is very jealous. If you send a command, you don't need to worry about specific things. You just need to give it back when someone else (the system) helps him.
• The event separator waits for the completion of the read event (compared to the reactor );
• When the event separator silently waits for the completion of the task, the operating system is already working. It reads data from the target, stores the data in the cache provided by the user, and finally notifies the event separator, I have finished this;
• The event sharer notifies the previous event handler that the thing you ordered is handled;
• The event handler will find that the data to be read has been placed in the cache provided by the handler, and everything can be done. If necessary, the event handler initiates another write operation as before, just like the previous steps.

Current practice

Open-source C ++ development framework ace [1,

3] (Douglas Schmidt, et al. development) provides a large number of independent underlying concurrency support classes (threads, mutex volumes, etc ). at the same time, it also provides several independent C ++ classes at the higher layer to implement the reactor and proactor modes. Although they are platform-independent units, they all provide different interfaces.

Ace proactor outperforms both performance and robustness in MS-Windows, mainly because Windows provides a series of efficient underlying asynchronous APIs. [4,

5].

(This may be outdated.) Unfortunately, not all operating systems provide robust support for underlying Asynchronization. For example, many UNIX systems are troublesome. therefore, Ace reactor may be a more suitable solution for UNIX systems. because the underlying support of the system is different, developers have to maintain several independent codes for better performance on each system: ace proactor for Windows and ACE reactor for Unix.

As we have mentioned, the real asynchronous mode requires support at the operating system level. It is very difficult to design a universal external interface for reactor and proactor due to the differences between event handlers and operating system interactions. This is also the difficulty in designing a universal development framework.

Better solutions

In this article, we will try to provide a solution that integrates the proactor and reactor modes. to demonstrate this scheme, we will slightly adjust the reactor and simulate it into an asynchronous proactor model (mainly to complete the actual read/write work performed by the event handler in the event splitter, we call this method"Analog asynchronous"). The following example shows how the read operation is completed:

• The event handler claims to be interested in the read event and provides parameters for storing the result, such as the cache area and read data length;
• The debugger waits (for example, by using select ());
• When an event arrives (you can read it), the debugger is awakened and the debugger executes non-blocking read operations (the previous event handler has given enough information ). After reading it, it notifies the event handler.
• The event handler is notified that the read operation has been completed and that it has the complete data to be obtained.

We can see that the reactor mode can be converted to the proactor mode by adding some features to the separator (also the above debuggers. All these operations are actually exactly the same as the reactor model application. We just need to scatter the work to different roles. In this way, there will be no additional overhead or performance loss. We can take a closer look at the two processes below and they actually did the same thing:

Standard classic reactor mode:

• Step 1) Wait for the event (reactor's work)
• Step 2) Send an "readable" event to a previously registered event handler or callback (what the reactor wants to do)
• Step 3) read data (what the user code needs to do)
• Step 4) process data (what the user code needs to do)

Simulated proactor mode:

• Step 1) Wait for the event (proactor's work)
• Step 2) read data (see, it turns into letting proactor do this)
• Step 3) Send the prepared message to the user's handler, that is, the event handler)
• Step 4) process data (what the user code needs to do)

In an operating system that does not support underlying asynchronous I/O APIs, this method can help us hide the differences between socket interfaces (whether it is performance or other) and provide a fully available and unified"Asynchronous interface". In this way, we can develop an independent universal interface for the platform.

Tproactor

The proposed tproactor solution has been implemented by terabepo/L [6. it has two implementations: C ++ and Java. c ++ uses the independent underlying components of the ACE platform, and finally provides a unified asynchronous interface on all operating systems.

The most important components in tproactor are engine and waitstrategy. the engine is used to maintain the lifecycle of asynchronous operations, while waitstrategy is used to manage concurrency policies. waitstrategy and engine are usually paired and provide a good matching interface between them.

Engines and wait policies are designed to be highly composable (see Appendix 1 for a complete implementation list ). Tproactor is a highly configurable solution by using asynchronous kernel APIs and synchronous UNIX APIs (Select (),
Poll (),/Dev/poll (Solaris 5.8 + ),Port_get(Solaris 5.10), realtime (RT) signals (Linux 2.4 +), epoll (Linux 2.6), K-queue (FreeBSD), it implements three internal engines (posix aio, sun AIO and emulated AIO) and hide the six types of wait policies. Tproactor implements the same interface as the standard ace proactor. In this way, it is possible to provide a cross-platform solution with only one piece of code for different platforms.

Engines and waitstrategies can be freely combined like Lego blocks. developers can choose an appropriate internal mechanism (engine and wait Policy) by configuring parameters at runtime ). You can configure the number of connections, system scalability, and operating system as needed. If the system supports the corresponding asynchronous underlying API, developers can select the real asynchronous policy. Otherwise, users can select the simulated asynchronous mode. We don't need to pay much attention to the Implementation Details of all these policies. What we see is an available asynchronous model.

For example, if an HTTP server running on Sun Solaris needs to support a large number of connections, engines such as/dev/poll or port_get () are suitable; if high throughput is required, it would be better to use the basic select () engine. Due to the inherent algorithm problems of different selection policies, elastic selection such as this is not available in the standard ace reactor/proactor mode (see appendix 2 ).

In terms of performance, our tests show that the simulation of asynchronous mode does not cause any overhead and does not slow down, but performance is improved. According to our test results, tproactor has a performance improvement of about 10-35% on Unix/Linux systems compared with the label ace reactor, while on Windows (throughput and response time are tested ).

Performance Comparison (Java/C ++/C #).

In addition to C ++, we also implemented tproactor. jdk1.4 in Java. Java only provides synchronization methods, such as select () [7,

8]. Java tproactor is based on Java's non-blocking function (Java. Nio package), similar to C ++'s tproactor using the select () engine.

Figure 1 and 2 show the transmission speed in bits/sec and the number of connections. These figures compare the echo servers implemented in the following three methods: Standard ace reactor implementation (based on RedHat Linux9.0), tproactor C ++/JAVA Implementation (Microsoft Windows platform and RedHat v9.0 ), and C # implementation. During the test, the three servers were connected using the same client and sent a fixed packet continuously.

These tests are performed on the same hardware, and the comparison of the results on different hardware is similar.

Figure 1. Windows XP/P4 2.6 GHz hyperthreading/512 mb ram. Figure 2. Linux Redhat 2.4.20-SMP/P4 2.6 GHz hyperthreading/512 mb ram.

User code example

The following is the echo server code framework implemented by tproactor Java. In general, developers only need to implement two interfaces: opread, which provides the cache for storing read results, and opwrite, which provides the cache for storing data to be written. At the same time, developers need to call back onreadcomplated () and onwritecompleted () to implement Protocol-related business code. These callbacks will be called when appropriate.

 class EchoServerProtocol implements AsynchHandler{   AsynchChannel achannel = null;   EchoServerProtocol( Demultiplexor m,  SelectableChannel channel )   throws Exception  {    this.achannel = new AsynchChannel( m, this, channel );  }   public void start() throws Exception  {    // called after construction    System.out.println( Thread.currentThread().getName() + ": EchoServer protocol started" );    achannel.read( buffer);  }   public void onReadCompleted( OpRead opRead ) throws Exception  {    if ( opRead.getError() != null )    {      // handle error, do clean-up if needed      System.out.println( "EchoServer::readCompleted: " +       opRead.getError().toString());      achannel.close();      return;    }     if ( opRead.getBytesCompleted () <= 0)    {      System.out.println("EchoServer::readCompleted: Peer closed "    + opRead.getBytesCompleted();      achannel.close();      return;    }     ByteBuffer buffer = opRead.getBuffer();     achannel.write(buffer);  }   public void onWriteCompleted(OpWrite opWrite)   throws Exception  {    // logically similar to onReadCompleted    ...  }}

Conclusion

Tproactor provides a general, flexible, and configurable high-performance communication component for multiple platforms. All the problems mentioned in Appendix 2 are well hidden in the internal implementation.

From the figure above, we can see that C ++ is still the best choice for writing high-performance servers, although Java is closely followed. However, due to Java implementation problems, it is not doing well on Windows (this should be a historical one ).

It should be noted that the above tests for Java are all in the form of raw data, and data processing is not involved (affecting performance ).

Looking at the rapid development of AIO in Linux [9], we can predict that the Linux kernel API will provide a large number of more robust asynchronous APIs, in this way, the new engine/Wait policy implemented based on this will easily solve the usability issues, and this will also benefit the standard ace proactor interface.

Appendix I

Engines and wait policies implemented in tproactor

Engine type	Wait Policy	Operating System
Posix_aio (true async) aio_read()/aio_write()	aio_suspend() Waiting for RT signal Callback function	POSIX complained Unix (not robust) POSIX (not robust) Sgi irix, Linux (not robust)
Sun_aio (true async) aio_read()/aio_write()	aio_wait()	Sun (not robust)
Emulated async Non-blockingread()/write()	select() poll() /Dev/poll Linux RT Signals Kqueue	Generic POSIX Mostly all POSIX implementations Sun Linux FreeBSD

Appendix II

All synchronization wait policies can be divided into two groups:

• Edge-triggered (e.g. Linux real-time signal)-signal readiness only when socket became ready (changes state );
• Level-triggered (e.g.select(),poll(),/Dev/poll)-readiness at any time.

Let's take a look at some common logic problems in the two groups:

• Edge-triggered group: After executing I/O operation, the demultiplexing loop can lose the state of socket readiness. example: The "read" handler did not read whole chunk of data, so the socket remains still ready for read. but the demultiplexor loop will
  Not receive next notification.
• Level-triggered group: When demultiplexor loop detects readiness, it starts the write/read user defined handler. but before the start, it shoshould remove socket descriptior from theset of monitored descriptors. otherwise, the same event can be dispatched
  Twice.
• Obviusly, solving these problems adds extra complexities to development. all these problems were resolved internally within tproactor and the developer shocould not worry about those details, while in the synch approach one needs to apply extra effort
  Resolve them.

Resources

[1] Douglas C. Schmidt, Stephen D. Huston "C ++ network programming." 2002, Addison-Wesley ISBN 0-201-60464-7

[2] W. Richard Steven s "UNIX network programming" Vol. 1 and 2, 1999, Prentice Hill, ISBN 0-13-490012-x

[3] Douglas C. Schmidt, Michael Peng, Hans rohnert, Frank buschmann "pattern-Oriented Software Architecture: Patterns for concurrent and networked objects, Volume 2" Wiley & Sons, NY 2000

[4] info: Socket overlapped I/O versus blocking/non-blocking mode. q181611. Microsoft Knowledge Base articles.

[5] Microsoft msdn. I/O Completion Ports.
Http://msdn.microsoft.com/library/default.asp? Url =/library/en-US/fileio/fs/I _o_completion_ports.asp

[6] tproactor (ACE compatible proactor ).
Www.terabit.com. au

[7] javadoc java. NiO. Channels
Http://java.sun.com/j2se/1.4.2/docs/api/java/nio/channels/package-summary.html

[8] javadoc java. NiO. channels. SPI class selectorprovider
Http://java.sun.com/j2se/1.4.2/docs/api/java/nio/channels/spi/SelectorProvider.html

[9] Linux AIO Development
Http://lse.sourceforge.net/io/aio.html, and
Http://archive.linuxsymposium.org/ols2003/Proceedings/All-Reprints/Reprint-Pulavarty-OLS2003.pdf

More

Ian Barile "I/O multiplexing & scalable socket servers", 2004 February, ddj

Further reading on event handling
-Http://www.cs.wustl.edu /~ Schmidt/ACE-papers.html

The adaptive communication environment
Http://www.cs.wustl.edu /~ Schmidt/ace.html

Terabit Solutions
Http://terabit.com.au/solutions.php