Device io (synchronous asynchronous Io, iocp/epool, etc.), and iocp and socket operations (criticism | discussion)

Iocp/epool is used to receive notifications when communicating with devices.

 

The files that the device accesses to the hard disk are related to the file system driver, while the socket accesses are related to the Network Driver (software> hardware.

 

The operating mode of the OS driver is generally to send a command, wait for the command to complete and obtain the result.

 

The level is similar.

User

-----------------------

Kernel <=> driver

 

There can be a number of drivers. The specific implementation has nothing to do with the user. The kernel will know when the operation is actually completed.

Two access modes with a long history

 

A. When the access is blocked (Read/send), after each API call, it will wait until the operation is complete and then return. At this time, the user thread is suspended. A waste of processing time.

Its working mode (the actual notification mode may be different from the description, but the process is like this)

User-> API-> send a request to the kernel to suspend the user thread

Request completed

The driver sends a request result notification to the kernel. The kernel returns the result to the API to restore the user thread.

 

B. asynchronous access:

User-> API queries the read/write status of a device based on user operations, and returns results based on the status (data is returned successfully or block is required for failure)

 

The special example here is reading.

For example, when you access a disk file, the first read request certainly has no data, because the kernel does not know which data you are requesting to read. It is impossible to open the file, the system immediately reads the file into the cache.

Therefore, for the first access to the file, the actual operation is to send the READ command to the driver, and then return the wocould block.

The next read operation can be completed immediately after the kernel receives the returned data.

 

For devices (such as sockets and serial ports) that can actively generate data, the first read operation may be completed immediately.

 

Certificate -----------------------------------------------------------------------------------------------------------------------------------------

In case of intensive devices:

The synchronization mode is obviously very inefficient, and most of the CPU time is waiting for the device to complete the operation.

The asynchronous mode is slightly better, but it requires a lot of time polling and many devices. When I/O is low, the CPU consumption is very high in the query.

 

The selecl/pool mode is used to query the read/write status of a bunch of devices. Then Mark readable or writable equipment for user operation.

 

In select/pool, a large number of device handles (R/W/E handle array) need to be submitted each time, but only one of them may be returned, resulting in low operation efficiency.

So epool appears.

 

The difference between epool and select/pool is that the list of devices it focuses on is maintained by the system kernel using epool_ctrl to increase or decrease the status of the devices or the status to be queried.

Therefore, you do not need to submit a device list for each query. Efficiency is greatly increased.

But epool

 

Iocp uses another completely different notification method.

When accessing the device through iocp, directly submit the command + user buffer to the kernel driver, and then return immediately. At this time, the user thread can continue to run and do other things.

When you want to know the command completion status, you can use getqueuedcompletionstatus to obtain the completed request and data.

Therefore, you can submit a large number of IO requests first, and then process and return them slowly.

 

The reason why iocp is more efficient than epoll is that, after each epool query, a status array is returned, and most of the data may be useless, but applications still find useful States through loops.

However, every time iocp returns something useful, users can directly use the result to reduce the search for this step. Therefore, the validity rate of a large number of devices is higher than that of the epool.

 

Other Io notification methods:

Because kqueue has never been used, I dare not comment on it.

 

Certificate ----------------------------------------------------------------------------------------------------------------------------------------------

Application of iocp on socket server

 

First, iocp has nothing to do with threads. It only deals with devices.

The so-called iocp work thread statement is inherently problematic. It should be thread worked with iocp, that is, the IO + logic thread.

NumberofconcurrentthreadsIt refers to the maximum number of threads that can query the iocp status at the same time. All threads with this number are suspended at getqueuedcompletionstatus.

 

Second, the balance between IO speed and processing speed.

Assume that a server program can accept 1 Mbit/s of data requests, but the computing process on its CPU can only process KB of data,

The final result of simply improving Io efficiency is buffer overflow and server crash.

 

Now let's take a look at the two methods of using iocp socket on the server.

A. 1 iocp object,Numberofconcurrentthreads= Ncpu * 2, threads workd with iocp = ncpu * 2.

The iocp recommendation method in msdn is also implemented on the Code project.

There is no problem with this mode, but it may cause efficiency loss due to misunderstandings and incorrect usage.

 

In the example in msdn, The iocp is a file operation, which is slightly different from the socket operation.

For example, file encryption algorithms. The file can be divided into small pieces of equal size, and then the read request is submitted, the data is obtained, and the write request is submitted. These are all completed in threads workd with iocp.

Because there is no pre-correlation between the data, the thread can be fully processed concurrently, which is the most efficient at this time. And automatically balances the IO and processing speed.

At the same time, when the IO speed is low, the Awakened thread may always be the first (iocp internally traces the thread mark and knows which thread is recently awakened ). Minimum thread switching overhead.

This is ideal for servers.

The reality is cruel. For a socket operation, because the received data is highly correlated, two threads may obtain two segments of the same connection to read data, in this case, the next thread can only wait for the previous thread to complete the operation, and then lock the connection for further processing.

At the same time, the data arriving at each time may not be a complete data block that can be processed. At this time, another read request is initiated. Until the complete data segment is obtained and the package is processed.

This type of model code is complex. If the application method is correct, it is a good choice.

 

 

Incorrect usage:

Some application methods use this entire model as an I/O thread, and then one or several logic threads process the input buffer separately. The problem is that the logic processing time is 0, io efficiency is maximized. If the logic thread cannot keep up with the I/O speed, the server will eventually be crash. (The buffer has been increasing)

 

B. N iocp objects, n threads, and socket are evenly bound to each iocp object

When n = 1, select/pool/epool is used. A single thread does not have packet sequence problems, and the efficiency is affected by the logic processing speed.

When n> 1, in intensive Io scenarios, less packet sequence checks seem to be more efficient than method A. However, if more lock operations are performed, the efficiency is reduced. There is only one thread, and other IO will not be able to handle it when waiting.

At the same time, thread switching overhead is high when there are few requests.

 

This model is best used as a separate Io thread. Only one thread is used for special Io, and other threads are used for logical data processing.

At this time, the logic processing time of the IO thread is basically unavailable, and the IO efficiency is maximized.

 

In addition, server is also very useful for simple protocols such as ECHO/socks.

 

The following two methods are summarized:

A. N (iocp + logic thread)

B. iocp I/O threads: 1 + N logical threads

 

Test:

Test Environment

Server is the same as client: win7 64 + 8g, M network. CPU I5 2500 KB

Server is a simple login + echo protocol. The request content of the ECHO client after the login request is successful.

After the client successfully logs in, the client sends 20 bytes of request content every MS, of which 4 bytes are the header of the custom protocol.

Tcp_nodely 1 is used for both sides to simulate a large number of IP packets.

5 K connections.

It is preliminarily estimated that the unidirectional traffic is (20 + 20) * 10 * connections. = 2 MB 16 Mbits

Actual traffic is 56 Mbit

Server:

Implemented in method A. The CPU consumption of 4 threads is about 20%, of which about 3/4 is the kernel time.

The single-thread implementation in Method B (I/O and logic are one thread together) is similar to the above. It seems that CPU consumption is low, and it should be related to thread switching.

 

That is to say, if iocp is used by a separate Io thread, one thread is the optimal solution, and other methods only reduce the efficiency.

The read requests of Io threads should be controlled by logic threads. Io threads only control the read requests when a group of packages is required. Avoid Buffer Overflow.

Write requests can be maintained by the IO thread (or counted only) to avoid Buffer Overflow.

 

Add some mistakes that are easy to make with iocp:

1. acceptex

Acceptex does not pass additional read buffer (recvdatalength = 0) to avoid DoS attacks.

(The malicious client does not send data to the connection. At this time, I/O is not completed, and the new accpetex cannot be shipped. Other clients cannot connect after consumption)

2. When sending/Recv/read/write directly returns a successful result, do not recycle the buffer contex. Successful call of iocp will be notified.

Only when the IO call fails must be recycled by yourself, And the rest should be processed in the iocp result.

3. overlapped and wsbuf should not be put together. nobuf errors may occur. After testing, this statement is unreasonable.

Wsae_nobuf consumes the system memory because of too many pending Io calls. It has nothing to do with Memory Locking.

The only possible optimization is the Memory Page alignment.