Windows IOCP Practices

about Windows IOCP

Some say Windows IOCP is the best thing on Windows. IOCP is a true asynchronous Io, which means that each time an IO request is initiated, the call itself returns immediately, while the copy of the IO operation and data from the kernel buffer to the user buffer is completed by the system until the process is closed before the user process is notified. There is no such asynchronous IO on Linux.

Use of IOCP

1. Create a new completion port. The completion port is designed to work with a thread pool, and threads from the thread pool are concurrently used to process the completed IO notifications. CreateIoCompletionPortThis API is used to create IOCP, and the last parameter is to specify the number of threads in the thread pool, generally taking CPU * 2, which maximizes the use of multicore CPUs and reduces the switching between threads.CreateIoCompletionPort(INVALID_HANDLE_VALUE, NULL, 0, dwNumberOfConcurrentThreads)
2. Creates a worker thread.
3. Associate an IO device to the completion port. Also called CreateIoCompletionPort (API design is a bit too casual, is there any historical reason?) ）。
   HANDLE h = CreateIoCompletionPort(hDevice, hCompletionPort, dwCompletionKey, 0);
4. Use overlapped IO, such as socket wsarecv/wsasend, even AcceptEx and ConnectEx. These calls simply initiate an IO request and then return immediately. A function call needs to initialize a OVERLAPPED struct, followed by a reference to its function.
5. The worker thread is a loop that blocks on the GetQueuedCompletionStatus call. GetQueuedCompletionStatusA completion packet is removed from the IO completion queue upon return. When thread pool threads are blocked, it is the responsibility of the system to complete the dispatch.

Some data structures within the IOCP

1. Device list: Contains a list of all devices associated with the completion port.
2. I/O completion Queue (FIFO): When an asynchronous IO request is completed, the system checks whether the IO device is associated with any IO completion port, and if so, adds a completion packet at the end of the IO completion port queue (FIF O in the order of the GetQueuedCompletionStatus queues), is waiting on this queue.
3. IOCP The associated thread waits for a queue: When a thread in a thread pool is called GetQueuedCompletionStatus , it is placed in a wait queue, and the IO completion port kernel object knows which threads are waiting to process completion packet based on this queue. The thread waits for the queue to be queued in a LIFO manner, that is, when a completion packet arrives, the system wakes up the thread that last called GetQueuedCompletionStatus into the waiting queue.

Interaction of IOCP and thread pool

1. Any thread can be called GetQueuedCompletionStatus to associate with an IO completion port, but a thread can only associate one IOCP, and when the thread exits or specifies a different IOCP or closes IOCP, the line friend with this IOCP unlocks the binding.
2. A concurrency value is specified when the IOCP is created, although any thread can be associated to this IOCP, but the concurrency value limits the number of threads that can run concurrently. Suppose such a scenario has a IOCP with a concurrency value of 1, but more than one thread is associated to this IOCP, and if there is always a completion packet waiting in the completion queue, when the running thread is called, it GetQueuedCompletionStatus returns immediately, and the thread finishes processing this Completion packet again GetQueuedCompletionStatus and returns immediately. During the process of completion packet, although there is always completion packet to be processed in the completion queue, the system does not dispatch other threads to execute because of the concurrency value of 1, although more than one thread is associated with IOCP. It also avoids the overhead of thread switching, because this thread is always executing.
3. In the above scenario, it looks like the other threads associated with the thread pool are useless, but it does not take into account the fact that the running thread is in a waiting state or is unbound from the IOCP for some reason. If a running thread is called Sleep , WaitFor* or a synchronous IO function, or any function that causes the current thread to change from a running state to a wait state, IOCP dispatches the other associated threads immediately, maintaining that one thread is always running.

IOCP problems encountered during the use of the

1. Because multithreading is involved is more complex than epoll + single-threaded programming.
2. The API is poorly designed, which also increases the difficulty of coding.
3. The document description is not clear, even without an official sample program, unofficial documents or programs are more or less wrong, it is difficult to use with ease. As an example of WSARecv, MSDN describes if WSARecv can return immediately, the return value is 0. Does this mean that the program has to handle IO completion in two places, one in which the IO returns immediately, and one is the worker thread GetQueuedCompletionStatus waiting for the IOCP to complete the queue. This is true for almost all asynchronous IO functions. But fortunately, it seems that even if you return 0 at once, there will be a completion packet in the completion queue, so there is no error in waiting for IO to complete in the completion queue in the worker thread.
4. The General network program that uses TCP to communicate, because the TCP flow unbounded nature, will be customized to such an application network packet: The first few bytes represents the length of the package, followed by the package's real content. When the application is unpacking, the corresponding packet length is obtained, and then the packet data of the corresponding length is intercepted. Such a process can be difficult in multi-threaded IOCP, multiple threads take different parts of each packet, and because completion packet's out-of-order does not guarantee that the order of packets fetched by each thread has been lost. Therefore, there must be a way to solve the order of the packages, and the unpacking process needs to synchronize each thread. This undoubtedly makes the code more complex.
5. IOCP as an asynchronous Io, can be very convenient to initiate IO, but each time you launch IO must commit a user memory, before the completion of the IO memory must be locked, and you can no longer use. Of course this is not the IOCP problem, which is determined by the asynchronous IO feature.

An example program for sending and receiving TCP application protocol packages

1. The protocol package is defined as the first two bytes to save the package length Len, and the Len behind the packet header is the specific content of the package. In order to simplify the coding and to take advantage of some features of IOCP, it is decided to start only one worker thread to handle all IO completion operations, and both the package and the packet are non-blocking asynchronous calls.
2. The buffer that is submitted to the asynchronous IO is removed from a pre-allocated memory so that the memory used by the IO operation is controllable and there is no memory fragmentation and full use of memory.
3. Several of IOCP's core APIs are related to parameter completionkey, overlapped. In the program Completionkey can correspond to which socket to operate, the overlapped corresponds to the specific IO operation.
4. Only one homogeneous IO operation (read or write) is allowed at the same time at commit.

The code here, the server-side program is relatively simple, can be implemented and verified by themselves.