Createiocompletionport and completion port

From Chapter 8 of networking programming for Microsoft Windows

The "complete port" model is the most complex I/O model so far. However, if an application needs to manage a large number of sockets at the same time, this model can often achieve the best system performance!

Essentially, the completion port model requires that we create a Win32 completion port object to manage overlapping I/O requests through a specified number of threads, to provide services for completed overlapping I/O requests.

Before using this model, you must first create an I/O port object to manage multiple I/O Requests targeting any number of socket handles. To do this, call the createcompletionport function.
The function is defined as follows:

Handle createiocompletionport (
Handle filehandle,
Handle existingcompletionport,
Ulong_ptr completionkey,
DWORD numberofconcurrentthreads
);

Before in-depth discussion of each parameter, we should first note that this function is actually used for two distinct purposes:
1. Create a complete port object.
2. associate a handle with the completion port.

When creating a complete port at the beginning, the only parameter of interest is numberofconcurrentthreads (number of concurrent threads). The first three parameters are ignored. The numberofconcurrentthreads parameter is special in that it defines the number of threads allowed to be executed simultaneously on a completion port. Ideally, we want each processor to be responsible for running one thread separately to provide services for port completion and avoid too frequent thread "scenario" switching. If this parameter is set to 0, it indicates how many processors are installed in the system and how many threads are allowed to run at the same time! Use the following code to create an I/O completion port:

Hiocp = createiocompletionport (invalid_handle_value, null, 0, 0 );

The purpose of this statement is to return a handle. After a socket handle is assigned to the port, it is used to calibrate (reference) the port ).

1. Worker thread and completion port
After a port is successfully created, You can associate the socket handle with the object. However, before associating a socket, you must first create one or more "worker threads" to provide services for the completed port after I/O requests are delivered to the completed port object. At this time, we may wonder how many threads should be created to provide services for port completion? This is actually a complicated aspect of completing the port model, because the number of service I/O requests depends on the overall design of the application. One important thing to remember here is that the number of concurrent threads we specify when calling createiocompletionport is different from the number of worker threads we intend to create. Earlier, we suggested using the createiocompletionport function for each processor.
Specify the number of threads (the number of processors, the number of threads) to avoid frequent thread "scenario" switching activities, thus affecting the overall performance of the system. The numberofconcurrentthreads parameter of the createiocompletionport function explicitly indicates that only N worker threads can run at a time on a completion port. If the number of worker threads created on the port is greater than N, only n threads are allowed to run at the same time. But in fact, in a short period of time, the system may exceed this value, but it will soon be reduced to the value set in the createiocompletionport function in advance. So why is the number of actually created worker threads sometimes more than the number set by the createiocompletionport function? Is this necessary? As previously stated, this mainly depends on
Overall Design of the application. Assume that one of our worker threads calls a function, such as sleep or waitforsingleobject, but is in the paused (locked or suspended) state, then another thread is allowed to replace it. In other words, we want to execute as many threads as possible at any time. Of course, the maximum number of threads is set in the createiocompletionport call. In this way, if you predict that your thread may be temporarily paused, it is better to create a thread with more values than the numberofconcurrentthreads parameter of createiocompletionport so that the system's potential can be fully realized at that time. Once there are enough worker threads on the completion port to provide services for I/O requests, You can associate the socket handle with the completion port. This requires us to call the createiocompletionport function on an existing completion port and provide socket information for the first three parameters -- filehandle, existingcompletionport, and completionkey. The filehandle parameter specifies a socket handle to be associated with the completion port. The existingcompletionport parameter specifies an existing completion port. The completionkey parameter specifies the "Single Handle data" to be associated with a specific socket handle. In this parameter, an application can store any type of information corresponding to a socket. It is called "Single Handle data" because it only
Data associated with the socket handle. It can be used as a pointer to a data structure to save the socket handle. In that structure, it also contains the socket handle and other information related to that socket.

Based on what we have learned so far, we will first build a basic application framework. The following describes how to use the complete port model to develop an echo server application. In this program, we basically follow the steps below:

1) create a complete port. The fourth parameter is set to 0, indicating that only one worker thread can be executed at a time on the completion port.
2) determine the number of processors installed in the system.
3) create a worker thread and provide services for completed I/O requests based on the processor information obtained in step 2.
4) Prepare a listening socket to listen for incoming connection requests on port 5150.
5) use the accept function to accept incoming connection requests.
6) create a data structure to hold "Single Handle data" and store the accepted socket handle in the structure.
7) Call createiocompletionport to associate the new socket handle returned from accept with the completed port. Pass the single-handle data structure to createiocompletionport by using the completionkey parameter.
8) Start the I/O operation on an accepted connection. Here, we want to deliver one or more asynchronous wsarecv or wsasend requests on the new socket through the overlapping I/O mechanism. After these I/O requests are completed, a worker thread will provide services for the I/O requests and continue to process future I/O requests. Later, in step 3) experience this in the specified worker routine.
9) Repeat Step 5 )~ 8) until the server is suspended.

2. Completing ports and overlapping I/O
After the socket handle is associated with a complete port, you can use the socket handle as the basis for sending and receiving requests, and start to process I/O requests. Next, you can start to rely on the completion port to receive notifications about the completion of I/O operations. Essentially, the complete port model utilizes the Win32 overlapping I/O mechanism. In this mechanism, Winsock API calls such as wsasend and wsarecv will return immediately. At this point, our application is responsible for receiving the call results at a later time through an overlapped structure. To do this in the completed port model, you need to use the getqueuedcompletionstatus function to wait for one or more worker threads on the finished port. The function is defined as follows:

Bool getqueuedcompletionstatus (
Handle completionport,
Lpdword lpnumberofbytes,
Pulong_ptr lpcompletionkey,
Lpoverlapped * lpoverlapped,
DWORD dwmilliseconds
);

The completionport parameter corresponds to the completion port to be waited on. The lpnumberofbytes parameter is responsible for receiving the actual number of bytes after an I/O operation (such as wsasend or wsarecv. The lpcompletionkey parameter returns "Single Handle data" for the socket originally passed into the createiocompletionport function ". As we mentioned earlier, we 'd better save the socket handle in this key. The lpoverlapped parameter is used to receive overlapping results of completed I/O operations. This is actually a very important parameter, because it can be used to obtain data for each I/O operation. The last parameter, dwmilliseconds, is used to specify the time when the caller wants to wait for a completed data packet to appear on the completed port. If it is set to infinite, the call will wait endlessly.

3. Single Handle data and single I/O operation data
After a worker thread receives an I/O completion notification from the getqueuedcompletionstatus API call, it contains necessary socket information in the lpcompletionkey and lpoverlapped parameters. With this information, you can continue the I/O processing on a socket by completing the port. Through these parameters, you can obtain two important socket data: Single Handle data and single I/O operation data. The lpcompletionkey parameter contains "Single Handle data", because when a socket is first associated with the completion port, the data corresponds to a specific socket handle. The data is passed through the completionkey parameter when calling the createiocompletionport API. As described earlier, an application can pass any type of data through this parameter. Normally, the application stores the socket handle related to the I/O Request here. The lpoverlapped parameter contains an overlapped structure followed by "single I/O operation data ". When our worker thread processes a complete data packet (it forwards the data intact, accepts the connection, ships the data to another thread, and so on), this information must be known. Single I/O operation data can be any number of bytes appended to the end of an overlapped structure. If a function requires an overlapped structure, we must pass such a structure to meet its requirements. To do this, a simple method is to define a structure and use the overlapped structure as the first element of the new structure. For example, you can define the following data structure to manage single I/O operation data:

Typedef struct
{
Overlapped;
Wsabuf databuf;
Char buffer [data_bufsize];
Bool operationtype;
} Per_io_operation_data

This structure demonstrates some important data elements that are usually associated with I/O operations, such as the type of the I/O operation just completed (sending or receiving requests ). In this structure, we think that the data buffer for completed I/O operations is very useful. To call a Winsock API function and assign it an overlapped structure, you can "shape" Your Own structure as an overlapped pointer, you can also simply unreference the overlapped element in the structure. For example:

Per_io_operation_data periodata;
// Call a function as follows
Wsarecv (socket,..., (overlapped *) & periodata );
// Or something like this
Wsarecv (socket,..., & (periodata. overlapped ));

In the later part of the worker thread, after the getqueuedcompletionstatus function returns an overlapping structure (and completion Key), you can unreference the operationtype member by unreferencing it, investigate which operation is delivered to the handle (you only need to shape the returned overlapping structure as your per_io_operation_data structure ). One of the biggest advantages of a single I/O operation data is that it allows us to manage multiple I/O operations (read/write, multiple reads, multiple writes, and so on ). At this time, you may have the following question: is it really necessary to deliver multiple I/O operations simultaneously on the same socket? The answer is the system's "scalability" or "scalability ". For example, if our machine is installed with multiple central processors, and each processor is running a worker thread
There may be several different processors on the same socket to send and receive data.

The last detail is how to properly close the I/O completion port-especially when one or more threads are running at the same time, when I/O operations are performed on several different sockets. An important problem to avoid is to forcibly release an overlapped structure while performing overlapping I/O operations. To avoid this situation, the best way is to call the closesocket function for each socket handle, and any overlapping I/O operations that have not been performed will be completed. Once all socket handles are closed, all worker threads must be terminated on the finished port. To do this, you need to use the postqueuedcompletionstatus function to send a special completion packet to each worker thread. This function indicates that each thread "ends immediately and exits ". The following is the definition of the postqueuedcompletionstatus function:

Bool postqueuedcompletionstatus (
Handle completionport,
DWORD dwnumberofbytestransferred,
Ulong_ptr dwcompletionkey,
Lpoverlapped
);

The completionport parameter specifies the completion port object to which the packet is sent. For the dwnumberofbytestransferred, dwcompletionkey, and lpoverlapped parameters, each of them allows us to specify a value and pass it directly to the corresponding parameter in the getqueuedcompletionstatus function. In this way, after a worker thread receives the three getqueuedcompletionstatus function parameters passed, it can decide when to exit based on the special values set by one of the three parameters. For example, the value 0 can be passed using the dwcompletionport parameter, and a worker thread will interpret it as a abort command. Once all worker threads are closed, you can use the closehandle function to close the completion port and exit the program safely.

Note: usage of functions such as createiocompletionport, postqueuedcompletionstatus, and getqueuedcompletionstatus.

Platform SDK: Storage

 

 

I/O Completion Ports

I/O Completion Ports are the mechanisms by which an application uses a pool of threads that was created when the application was started to process asynchronous I/O requests. these threads are created for the sole purpose of processing I/O requests. applications that process logs concurrent asynchronous I/O requests can do so more quickly and efficiently by using I/O Completion Ports than by using creating threads at the time of the I/O Request.

 

 

 

 

 

I/O completion port (s) is a mechanism through which an application will first create a thread pool at startup, the application then uses the thread pool to process asynchronous I/O requests. These threads are created for the sole purpose of processing I/O requests. For applications that process a large number of concurrent asynchronous I/O requests, the completion port (s) is used when a thread is created when an I/O Request occurs) it can be faster and more efficient.

 

 

 

 

 

The createiocompletionport function associates an I/O completion port with one or more file handles. when an asynchronous I/O operation started on a file handle associated with a completion port is completed, an I/O completion packet is queued to the port. this can be used to combine the synchronization point for multiple file handles into a single object.

 

 

 

 

 

The createiocompletionport function associates an I/O completion port with one or more file handles. When the asynchronous I/O operations started on a file handle related to the completed port are completed, an I/O completed package will enter the queue of the completed port. For multiple file handles, these file handles can be combined into a single object, which can be used to combine synchronization points?

 

 

 

 

 

A thread uses the getqueuedcompletionstatus function to wait for a completion packet to be queued to the completion port, rather than waiting directly for the asynchronous I/O to complete. threads that block their execution on a completion port are released in last-in-first-out (LIFO) Order. this means that when a completion packet is queued to the completion port, the system releases the last thread to block its execution on the port.

 

 

Call the getqueuedcompletionstatus function. A thread waits for a complete package to enter the queue of the completed port, instead of waiting for the asynchronous I/O Request to complete. Threads (s) will block their running on the completion port (the queue is released in the FIFO order ). This means that when a finished package enters the queue of the finished port, the system will release the thread that has recently been blocked on the finished port.

 

 

 

 

 

When a thread CILS getqueuedcompletionstatus, it is associated with the specified completion port until it exits, specifies a different completion port, or frees the completion port. A thread can be associated with at most one completion port.

 

 

When getqueuedcompletionstatus is called, the thread will establish a connection with a specified completion port, continuing the existence cycle of the thread, or being specified with a different completion port, or release the connection with the completion port. A thread can only be associated with up to one completion port.

 

 

 

 

 

The most important property of a completion port is the concurrency value. the concurrency value of a completion port is specified when the completion port is created. this value limits the number of runnable threads associated with the completion port. when the total number of runnable threads associated with the completion port reaches the concurrency value, the system blocks the execution of any subsequent threads that specify the completion port until the number of runnable threads associated with the completion port drops below the concurrency value. the most efficient scenario occurs when there are completion packets waiting in the queue, but no waits can be satisfied because the port has reached its concurrency limit. in this case, when a running thread callgetqueuedcompletionstatus, it will immediately pick up the queued completion packet. no context switches will occur, because the running thread is continually picking up completion packets and the other threads are unable to run.

 

 

The most important feature of port completion is the concurrency. The concurrency of the completed port can be specified when the completed port is created. This concurrency limit the number of runnable threads associated with the completion port. When the total number of runable threads associated with the completion port reaches this concurrency, the system will block any subsequent thread execution associated with the completion port, until the number of runable threads associated with the port is reduced to less than the number of concurrent threads. The most effective assumption is that there is a finished packet waiting in the queue, but not waiting to be satisfied, because at this time the finished port reaches the limit of its concurrency. At this time, when a running thread calls getqueuedcompletionstatus, it immediately removes the completion package from the queue. In this way, there is no environment switch, because the running thread will continuously remove the complete package from the queue, and other threads will not be able to run.

 

 

 

 

 

The best value to pick for the concurrency value is the number of CPUs on the machine. if your transaction required a lengthy computation, a larger concurrency value will allow more threads to run. each transaction will take longer to complete, but more transactions will be processed at the same time. it is easy to experiment with the concurrency value to achieve the best effect for your application.

 

 

The best choice for concurrency is the number of CPUs in your computer. If your transaction processing takes a long computing time, a large amount of concurrency can allow more threads to run. Although it takes longer to complete each transaction, more transactions can be processed at the same time. For applications, it is easy to obtain the best results by testing the concurrency.

 

 

 

 

 

The postqueuedcompletionstatus function allows an application to queue its own special-purpose I/O completion packets to the completion port without starting an asynchronous I/O operation. this is useful for policying worker threads of external events.

 

 

The postqueuedcompletionstatus function allows applications to queue custom dedicated I/O completion packages without starting an asynchronous I/O operation. This is useful for the worker thread that notifies external events.

 

 

 

 

 

The completion port is freed when there are no more references to it. the completion port handle and every file handle associated with the completion port reference the completion port. all the handles must be closed to free the completion port. to close the port handle, call the closehandle function.

 

 

When there are no more references for a completed port, you need to release the completed port. The completed port handle and all file handles associated with the completed port need to be released. Call closehandle to release the handle of the completed port.