Windows I/O completion port

Windows I/O completion port

Windows
18:17:42
Read 418

Comment 0


Font size: Large
Medium
JS-fcurrent fc05 "> small

Windows
Complete Port Programming

1. Basic Concepts
2. Features of Windows Port completion
3. Complete ports
) Related data structures and Creation
4. Working Principles of port threads
5. Example code of Windows Port completion

Complete Port Programming in Windows


Abstract: It is never easy to develop network programs, although only a few rules need to be followed: Create a socket, initiate a connection, accept a connection, send and receive data, and so on. Real difficulties
It allows your program to adapt from connecting thousands of connections to tens of thousands of connections. Using Windows to achieve port overlap I/O, you can easily
The Platform develops network service programs that support a large number of connections. This article introduces the basic principle of using the port model development on Windows platform, and provides the actual example. This article focuses on the C/S structure
Generally, developing a large-capacity and scalable Winsock program is a service program.

1. Basic Concepts

Set
Backup-anything that allows communication on Windows operating systems, such as files, directories, serial ports, parallel ports, mail slots, named pipelines, unnamed pipelines, sockets, consoles, logical disks, and physical
Disks. The vast majority of functions dealing with devices are createfile, readfile, writefile, etc,
So we can't see the ** file function, just think of the file device.
.

There are two ways to communicate with a device: synchronous and asynchronous.
Lower
When you call functions such as readfile, the function will wait for the system to finish the required work before returning; asynchronous
Lower
, Readfile and other functions will directly
Then, the system completes the operation on the device, and then notifies the device to complete the operation in some way.

Overlapping I/O
---- As the name implies, when you call a function (such as readfile)
When you immediately return and perform other operations on your own, the system is also performing the operations you requested on the I/0 device, during this period, your program and the internal actions of the system overlap, so it is better.
Yes. Therefore, overlapping I/O uses I/O devices in asynchronous mode. Overlapping I/O requires a very important data knot
Structure: overlapped
.

2. Features of Windows Port completion


Win32 overlapping I/O (overlapped
The I/O) mechanism allows you to initiate an operation and receive information after the operation is complete. For operations that take a long time to complete, the overlapping Io mechanism is particularly useful because the threads that initiate overlapping operations are
After a stacked request is sent, you can do other things freely. On WINNT and win2000, the truly scalable I/O model provided is to use the complete port (Completion
Port. The completion port is a Windows Kernel Object.
. When the completion port is used for asynchronous overlapping I/0, of course, overlapping I/O does not have to be used as the completion port. Similarly, the device kernel object, event object, and advertisement
I/0 can also be used.
However, the thread pool management is provided inside the port to avoid the overhead of repeatedly creating threads.
The number of threads can be flexibly determined based on the number of CPUs, and the number of thread scheduling times can be reduced to improve performance.
. Similar to wsaasyncselect and
The select function mechanism is easier to be compatible with Unix, but it is difficult to implement the desired "scalability ". In addition, the Windows Port completion mechanism has been optimized inside the operating system to provide
Higher efficiency. Therefore, we chose to complete port development for our server program.

1) The initiating operation may not be completed: the system will notify you when the operation is completed to process the operation result by waiting on the completed port. Therefore, there must be a thread that checks the port and obtains the operation result. On the completion port
The waiting thread system is optimized. Unless the thread being executed is blocked, no new threads will be activated to reduce the performance cost caused by thread switching. So if the program does not have too many blocking operations
It is necessary to start too many threads and use twice the number of cpus. Generally, such multithreading is enough.
.

2) binding methods of operations and related data: When submitting data, the user tags the data and records the operation type. When the user processes the operation result, by checking your own tag and Operating System
The results are processed accordingly.

3) method returned by the operation: after the operation is completed, the program should be notified for subsequent processing. However, the write operation does not notify the user. If the user's write operation cannot be completed immediately, the data related to the write operation will be saved to non-handed
During the buffer change, the system automatically releases the buffer when the operation is completed. After the write operation is initiated, the memory used can be released. However, if too many non-swap buffers are used, the system will stop responding.

3,
Complete the data structure and creation of port (Completion Ports)


In fact, we can regard the completed port as a queue maintained by the system. The operating system puts the Event Notifications of overlapping Io operations in this queue because they are exposed.
The Event Notification of "Operation completed", so the name is "completion port" (Completion
Ports ). After a socket is created, it can be associated with a complete port at any time.

Overlapped Data Structure

Typedef struct
_ Overlapped {
Ulong_ptr internal; // assigned internally by the system to indicate the system status

Ulong_ptr internalhigh; // assigned internally by the system, indicating the number of transmitted bytes
Union {

Struct {
DWORD offset;
// Synthesize a 64-bit integer with offsethigh to indicate the number of bytes in the file header.
DWORD
Offsethigh; // if it is not for file I/O operations, the offset must be set to 0
};

Pvoid pointer;
};
Handle hevent;
// If not, set it to 0; otherwise, assign a valid event handle.
} Overlapped, * lpoverlapped;

Lower
An example of using readfile Asynchronously

Overlapped
Overlapped;
Overlapped. offset = 345;
Overlapped. offsethigh = 0;
Overlapped. hevent = 0;

// Assume that all other parameters have been initialized
Readfile (hfile, buffer, sizeof (buffer), & dwnumbytesread, & overlapped );

In this way, the file read operation is completed asynchronously, And the readfile function returns the result, which is done by the operating system.

The following describes several functions related to the overlapped structure.

Functions waiting for overlapping I/0 operations to complete

Bool getoverlappedresult
(
Handle hfile,
Lpoverlapped
Lpoverlapped, // accept the returned overlapping I/0 structure
Lpdword lpcbtransfer, // number of bytes successfully transferred
Bool
Fwait // true: returns the result only when the operation is completed, and returns the result directly if the operation is not completed,

// By using the getlasterror () function, error_io_incomplete is returned.
);

While
Macro
Hasoverlappediocompleted
Can help us test whether overlapping I/0 operations are performed?
Complete. The macro tests the internal Member of the overlapped structure to check whether it is equal to the status_pending value.

Generally, an application can create multiple worker threads to process notification events on the port.
Work
The number of threads depends on the specific needs of the program. However, in ideal cases, a CPU
Create a thread. In the ideal port completion model, each thread can obtain an "Atomic" time slice from the system, and run and check the completed port in turn. Thread switching is an additional overhead. But in actual
During development, you should also consider whether these threads involve other blocking operations. If a thread is congested, the system suspends the thread so that other threads can get the running time. Therefore
You can create several threads to make full use of the time.

Function for port Creation

End
Cheng
A port is a kernel object. It is always associated with at least one valid device handle during use. A complete port is a complex Kernel Object. The function of creating a port is as follows:
Handle createiocompletionport
(
In handle
Filehandle,
In handle existingcompletionport,
In
Ulong_ptr completionkey,
In DWORD numberofconcurrentthreads

);

Generally, the creation process is divided into two steps:

Step 1: Create a New completed port kernel object. You can use the following
Function:
Handle createnewcompletionport (DWORD dwnumberofthreads)

{
Return
Createiocompletionport (invalid_handle_value, null, null, dwnumberofthreads );

};
Step 2: Associate the created port with a valid device handle. You can use the following function:

Bool assicoatedevicewithcompletionport (handle hcompport, handle
Hdevice, DWORD dwcompkey)
{
Handle
H = createiocompletionport (hdevice, hcompport, dwcompkey, 0 );

Return H = hcompport;
};
Description:

1) The createiocompletionport function can also create a complete port object and associate it with a valid device handle at a time.
2) completionkey is a parameter that can be defined by ourselves. We can assign the address of a structure to it and use it when appropriate.
,
It is best to ensure that the memory in the structure is not allocated to the stack, unless you are sure that the memory will be retained until the moment you want to use it.
3) numberofconcurrentthreads
This parameter is used to specify the maximum number of threads that can be run simultaneously. Generally, it is set to 0. The system automatically determines the number of threads based on the number of CPUs.
4) after the created and associated actions are completed, the system will finish
The device handle and completion key associated with the port are added to the device list of the completed port as a record. If you have multiple Completion Ports, there will be multiple corresponding device lists. If the device handle is disabled
The record is automatically deleted.

4. Working Principles of port threads

1) Complete the Port Management Thread Pool


The completion port can help us manage the thread pool, but the threads in the thread pool need to be created by using _ beginthreadex.
What notifications are sent to complete port management of our new thread?
The answer lies in the getqueuedcompletionstatus function. The function prototype:
Bool getqueuedcompletionstatus
(
In handle
Completionport,
Out lpdword lpnumberofbytestransferred,

Out pulong_ptr lpcompletionkey,
Out lpoverlapped * lpoverlapped,

In DWORD dwmilliseconds
);

This function tries to extract records from the specified completion port I/0 Completion queue. Only when the overlapping I/O actions are completed, there is a record in the Completion queue. All threads that call this function will be placed in the wait thread queue of the completion port. Therefore, the completion port can help us maintain this line in our own thread pool.
Cheng
. Completion port I/0 Completion queue stores the result when overlapping I/0 is completed ----
A record with four fields. The first three fields correspond to the 2, 3, and 4 parameters of the getqueuedcompletionstatus function. The last field is the error message.
Dwerror. We
You can also call
Postqueudcompletionstatus simulates an overlapping I/0 operation
.

When there is a record in the I/0 Completion queue, the completion port will check the waiting thread queue
The threads in the queue are added to the queue by calling the getqueuedcompletionstatus function.
And so on
The waiting thread queue is simple, but the IDs of these threads are saved. The completion port puts the ID of a thread queue into the release according to the principle of "back-in-first-out ".
In the thread list, the thread will change from the sleep status returned by the function waiting for getqueuedcompletionstatus to the scheduling status waiting for CPU scheduling.
.
Therefore, to make our threads complete port management, we must call the getqueuedcompletionstatus function. Out of performance optimization, port maintenance is actually completed
Protected A pause thread list. For details, refer to the Windows Advanced Programming Guide. We now know enough.

2) data transmission between threads


The most common way to complete data transmission between threads on the port is to pass parameters to the thread function in the _ beginthreadex function, or use global variables. However, the completion port also has its own data transmission method. The answer lies in the completionkey and overlapped parameters.
.
Completionkey
The device table that is saved in the completed port table corresponds to the device handle. You can save the data related to the device handle to the completionkey or
Completionkey is represented as a structure pointer, so that richer content can be transmitted. This content can only be done when the port and device handle are associated at the beginning, so it cannot be dynamically modified later.
Change.

The overlapped parameter is passed to the end of each call to a function such as readfile that supports overlapping I/0.
Port-based
. We can see that if we do not operate on the file device, the member variables of this structure will have almost no effect on us. We need additional information to create our own structure,
Then, the overlapped structure variable is used as the first member of our structure variable, and the address of the first member variable is passed to a function such as readfile. Yes, of course.
By compiling. When the getqueuedcompletionstatus function returns, we can get the address of the first member variable, and then a simple forced conversion, we can
Use it as a pointer to the complete custom structure, so that you can transmit a lot of additional data. Great! Only note that if cross-thread transmission is performed, the data should be distributed to the stack, and the receiving end should
This releases the data after it is used up. We usually need to put the buffer required by asynchronous functions such as readfile into our custom structure.
When getqueuedcompletionstatus is returned, I/0 operation data is stored in the buffer variable of our custom structure. The completionkey and overlapped parameters can both be passed through
Get the getqueuedcompletionstatus Function
.

3) Secure thread exit

Multithreading: To execute asynchronous data processing more than once, you need to use the following statement:
While
(True)
{
......
Getqueuedcompletionstatus
(...);
......
}
That
How does a thread exit?
The answer is the postqueudcompletionstatus mentioned above.
Letter
Number. We can send a custom structure address containing the overlapped member variable to it, which contains a status variable. When the status variable is the exit sign, the thread will execute the clear action and then
Exit.

5. Example code of Windows Port completion

DWORD winapi
Workerthread (lpvoid lpparam)
{
Ulong_ptr * perhandlekey;
Overlapped
* Overlap;
Overlappedplus * overlapplus;
Overlappedplus * newolp;
DWORD
Dwbytesxfered;
While (1)
{
Ret = getqueuedcompletionstatus
(Hiocp,
& Dwbytesxfered, (pulong_ptr) & perhandlekey, & overlap,
Infinite );
If (ret = 0)
{
// Operation failed

Continue;
}
Overlapplus = containing_record (overlap,
Overlappedplus, Ol );
Switch (overlapplus-> opcode)
{
Case
Op_accept:
// Client socket is contained in overlapplus. sclient
 
// Add client to completion port
Createiocompletionport
(Handle) overlapplus-> sclient,
Hiocp, (ulong_ptr) 0, 0 );
// Need a new overlappedplus Structure
 
// For the newly accepted socket. Perhaps
// Keep a look aside
List of free structures.
Newolp = allocateoverlappedplus ();
If
(! Newolp)
{
// Error
}
Newolp-> S =
Overlapplus-> sclient;
Newolp-> opcode = op_read;
// This
Function divpares the data to be sent
 
Preparesendbuffer (& newolp-> wbuf );
Ret = wsasend
(Newolp-> S, & newolp-> wbuf, 1,
& Newolp-> dwbytes, 0, & newolp. Ol, null );
If (ret =
Socket_error)
{
If (wsagetlasterror ()! = Wsa_io_pending)

{
// Error
}
}
// Put structure in look
Aside list for later use
Freeoverlappedplus (overlapplus );
//
Signal accept thread to issue another acceptex
Setevent
(Hacceptthread );
Break;
Case
Op_read:
// Process the data read
// Repost the read if
Necessary, reusing the same
// Receive buffer as before
 
Memset (& overlapplus-> ol, 0, sizeof (overlapped ));
Ret = wsarecv
(Overlapplus-> S,
& Overlapplus-> wbuf, 1, & overlapplus-> dwbytes,
& Overlapplus-> dwflags, & overlapplus-> ol, null );
If
(Ret = socket_error)
{
If (wsagetlasterror ()! =
Wsa_io_pending)
{
// Error
}
}
Break;
Case
Op_write:
// Process the data sent, etc.
Break;
}//
Switch
} // While
} // Workerthread

Check the preceding code. Note that if the overlapped operation fails immediately (for example, socket_error or other non-wsa_io_pending errors are returned ),
No completion notification time will be placed in the completion port queue. Otherwise, the corresponding notification time must be placed in the completion port queue. For more information about the complete port mechanism of WinSock, see
Microsoft platform SDK of msdn, where there is an example of port completion.