Title: design issues when using iocp in a Winsock server)

Title: design issues when using iocp in a Winsock Server
Title: Use iocp to develop some design content for the Winsock server.
Address: http://support.microsoft.com/kb/192800
Http://www.libing.net.cn/post/Design-Issues-When-Using-IOCP-in-a-Winsock-Server.php

You are welcome to repost it. Please retain the translator.
Translator: Hu zhangyou huzhangyou
MSN: huzhangyou2002 (AT) gmail.com
QQ: 3803308
Blog: http://www.libing.net.cn

Summary
This article assumes you already understand the I/O model of the Windows nt I/O completion port (iocp) and are familiar with the related APIs. if you want to learn iocp, please see advanced windows (3rd edition) by Jeffery Richter, chapter 15 device I/O for an excellent discussion on iocp implementation and the APIS you need to use it.

Translation: I have read this article before assuming that you have understood the I/O model of the Windows NT System and are familiar with related API functions. If you want to learn iocp, read chapter 15th of advanced windows (3rd edition) of Jeffery Richter. This section describes related API functions and discusses iocp implementation.

An iocp provides a model for developing very high performance and very scalable server programs. direct iocp support was added to Winsock2 and is fully implemented on the Windows NT platform. however, iocp is the hardest to understand implement among all Windows nt I/O models. to help you design a better socket server using iocp, a number of tips are provided in this Article.

A single iocp provides a highly efficient and Scalable Server program model for development. The iocp model is added to Winsock2 and is also implemented under the Windows NT platform. However, iocp is the most difficult to understand and implement in all Windows nt I/O models at the same time. To help you understand how to design an iocp server, the following key points may help you.

More information
Tip 1: Use Winsock2 iocp-capable functions, such as wsasend and wsarecv, over Win32 file I/O functions, such as writefile and readfile.

Translation:
Tip 1: Use iocp functions of Winsock2, such as wsasend and wsarecv, to replace Win32 file I/O functions, such as writefile and readfile.

Socket handles from Microsoft-based protocol providers are ifs handles so you can use Win32 file I/O CILS with the handle. however, the interactions between the provider and file system involve kernel/user mode transition, thread context switches, and parameter values als that result in a significant performance penalty. you shoshould use only Winsock2 iocp-capable functions with iocp.

Translation:
If you use Win32 file I/O function call, it will involve a lot of performance loss. For example, kernel/user mode conversion, thread switching, and parameter arrangement configuration. Therefore, we recommend that you use the iocp function of Winsock2.
The additional parameter parameters als and mode transitions in readfile and writefile only occur if the provider does not have xp1_ifs_handles bit set in dwserviceflags1 of its wsaprotocol_info structure.

Translation:
When the xp1_ifs_handles bit operations of wsaprotocol_info in the dwserviceflags1 structure are not set, the additional parameter arrangement configuration of readfile and writefile and modal switching take place.

Note: These providers have an unavoidable additional mode transition, even in the case of wsasend and wsarecv, although readfile and writefile will have more of them.

Tip 2: choose the number of the concurrent worker threads allowed and the total number of the worker threads to spawn.

Translation:
Tip 2: select the number of concurrent working threads.
The number of worker threads and the number of concurrent threads that the iocp uses are not the same thing. you can decide to have a maximum of 2 Concurrent threads used by the iocp and a pool of 10 worker threads. you have a pool of worker threads greater than or equal to the number of concurrent threads used by the iocp so that a worker thread handling a dequeued completion packet can call one of the Win32 "wait" functions without delaying the handling of other queued I/O packets.

Translation:
The number of worker threads used by iocp is not the same as the number of concurrent threads. You can use up to 2 Concurrent threads and 10 working threads at the same time. You can have many working threads. The number of concurrent threads is generally smaller than or equal to the number of working threads. A worker thread processes a queue containing the status of completed data packets. If there is no data, they will be waiting.
If there are completion packets waiting to be dequeued, the system will wake up another worker thread. eventually, the first thread satisfies it's wait and it can be run again. when this happens, the number of the threads that can be run is higher than the concurrency allowed on the iocp (for example, numberofconcurrentthreads ). however, when next worker thread CILS getqueuecompletionstatus and enters Wait Status, the system does not wake it up. in other words, the system tries to keep your requested number of concurrent worker threads.

Translation:
If a complete packet needs to be sent out of the queue, the system will wake up another thread. Finally, the first thread stops waiting and continues running. When this trigger occurs, the number of threads that can be executed is higher than the number of concurrent threads. However, when the worker thread calls getqueuecompletionstatus and enters the wait status, the system does not wake it up. In other words, the system tries its best to keep the number of concurrent working threads you require. (The translation here may not be fluent. Sorry)

Typically, you only need one concurrent worker thread per CPU for iocp. To do this, enter 0 for numberofconcurrentthreads in the createiocompletionport call when you first create the iocp.

Typically, you only need one concurrent thread for each CPU. Therefore, when calling the createiocompletionport function, set the numberofconcurrentthreads parameter to 0.

Tip 3: associate a posted I/O operation with a dequeued completion packet.

Getqueuedcompletionstatus returns a completion key and an overlapped structure for the I/O when dequeuing a completion packet. you shoshould use these two structures to return per handle and per I/O operation information, respectively. you can use your socket handle as the completion key when you register the socket with the iocp to provide per handle information. to provide per I/O operation "extend" the overlapped structure to contain your application-specific I/O-state information. also, make sure you provide a unique overlapped structure for each overlapped I/O. when an I/O completes, the same pointer to the overlapped I/O structure is returned.

Translation:
Tip 3: Associate an I/O operation with the same queue.
When dequeuing a completion packet is executed, the getqueuedcompletionstatus function returns a completion key and an overlapped structure. You should use these two structures to return each handle and information without I/O operations. After you register your socket and manage iocp, you can use your socket handle as the completion key. To provide extensions for each I/O operation, the overlapped structure can contain the specific I/O status information of your program. At the same time, you must ensure that you provide a unique overlapped structure for each overlapped I/O. When an I/O operation is completed, the same pointer pointing to the overlapped I/O operation will be returned.

Tip 4: I/O completion packet queuing behavior.

The order in which I/O completion packets are queued in the iocp is not necessarily the same order the Winsock2 I/O callwere made. additionally, if a Winsock2 I/O call returns success or io_pending, it is guaranteed that a completion packet will be queued to the iocp when the I/O completes, regardless of whether the socket handle is closed. after you close a socket handle, future callto wsasend, wsasendto, wsarecv, or wsarecvfrom will fail with a return code other than success or io_pending, which will not generate a completion packet. the status of the completion packet retrieved by getqueuedcompletionstatus for I/O previusly posted cocould indicate a failure in this case.

If you delete the iocp itself, no more I/O can be posted to the iocp because the iocp handle itself is invalid. however, the system's underlying iocp kernel structures do not go away until all successfully posted I/OS are completed.

Translation:
Tip 4: I/O completed package queue status

In iocp, the order in which the I/O completion package is waiting is not necessarily the same as that in Winsock2 I/O call. If a Winsock2 I/O call returns success or io_pending. After you close a socket handle, you can call wsasend, wsasendto, wsarecv, or wsarecvfrom to return results other than success and io_pending. They will not generate the completion package. An error occurs in the status package returned by the getqueuedcompletionstatus function.
Tip 5: iocp cleanup.

The most important thing to remember when invention Ming ICOP cleanup is the same when using overlapped I/O: do not free an overlapped structure if the I/O for it has not yet completed. the hasoverlappediocompleted macro allows you to detect if an I/O has completed from its overlapped structure.

Translation:
Tip 5: Clear the iocp.
Do not clean up unfinished ports. The hasoverlappediocompleted macro can help you determine whether the port has been completed.
There are typically two scenarios for shutting down a server. in the first scenario, you do not care about the completion status of outstanding I/OS and you just want to shut down as fast as you can. in the second scenario, you want to shut down the server, but you do need to know the completion status of each outstanding I/O.

Translation:
There are two ways to disable the server. In the first method, you do not need to care about the completion status.
In the second method, you need to disable the server and know the completion status.
In the first scenario, you can call postqueuecompletionstatus (n times, where N is the number of worker threads) to post a special completion packet that informs the worker thread to exit immediately, close all socket handles and their associated overlapped structures, and then close the completion port. again, make sure you use hasoverlappediocompleted to check the completion status of an overlapped structure before you free it. if a socket is closed, all outstanding I/O on the socket eventually complete quickly.

Translation:
The first method is to send postqueuecompletionstatus. The number is the number of worker threads.
In the second scenario, you can delay exiting worker threads so that all completion packets can be properly dequeued. you can start by closing all socket handles and the iocp. however, you need to maintain a count of the number of outstanding I/OS so that your worker thread can know when it is safe to exit the thread. the performance penalty of having a global I/o counter protected with a critical section for an iocp server is not as bad as might be expected because the active worker thread does not switch out if there are more completion packets waiting in the queue.

Method 2: You can delay exiting the working thread, so all the completion packages can be properly matched. You can close all socket handles and iocp. Therefore, you need to maintain a series of active I/OS, so your thread can know that those threads can exit safely.