[Go] understanding I/O completion Port

Original: http://dev.gameres.com/Program/Control/IOCP.htm

Also attached: http://stackoverflow.com/questions/5283032/i-o-completion-ports-advantages-and-disadvantages

Please read this IOCP tutorial. I will first give the definition of IOCP and then give its implementation method, and finally analyze an echo program for you to get rid of IOCP mystery cloud, remove your heart to IOCP trouble. OK, but I can't guarantee you know everything about IOCP, but I will try my best. Here are the techniques I'll mention in this article:
I/O port
Synchronous/Asynchronous
Clogged/non-clogging
Service Side/Client
Multithreaded programming
Winsock API 2.0

Before that, I had developed a project, one of which needed network support, and then took into account the portability of the code, as long as the use of Select,connect,accept,listen,send and recv, plus a few # The ifdef package was used to deal with the incompatibilities between the Winsock and BSD sockets [sockets], and a network subsystem was written out with very little code for a few hours, and it still makes me very memorable. It's been a long time since then.

Some days ago, we plan to do a network game, I take the initiative to undertake the network this piece, think this is not a small case, the heart of stealing music ah. Online games good, online games for hundreds of players to provide fun and a secret game experience, they fight each other online or join the team to overcome the common enemy. I am full of confidence to write my network, so, found that the past blocking synchronization mode mode can not get a huge number of multi-player [MMP] architecture, directly denied. So, there is a IOCP, if can be easily and lifting IOCP, also will not have this tutorial. Please follow me to get to the point below.


What is IOCP?
First, let's take a look at IOCP's evaluation.
The I/O completion port may be the most complex kernel object provided by Win32.
[Advanced Windows 3rd] Jeffrey Richter
This is the best way to implement a high-capacity network server [IOCP].
[Windows sockets2.0:write Scalable Winsock Apps Using completion Ports]
Microsoft Corporation
The completion port model provides the best scalability. This model is ideal for handling hundreds of or even thousands of sockets.
[Windows network Programming 2nd] Anthony Jones & Jim Ohlund
I/O completion ports are particularly important because they are the only technology that is suitable for high-load servers [which must maintain many connection lines at the same time]. Completion ports uses some threads to help balance the load caused by I/O requests. This architecture is particularly suitable for "scalable" servers that are produced in SMP systems.
[Win32 multithreaded Programming] Jim Beveridge & Robert Wiener


It seems that we have every reason to believe that IOCP is the first choice for large network architectures. What the hell is that IOCP?

Microsoft introduced the concept of IOCP in Winsock2. IOCP Full name I/O completion port, Chinese to I/O completion port. IOCP is an asynchronous I/O API that effectively notifies an application of I/O events. Unlike using Select () or other asynchronous methods, a socket [socket] is associated with a completion port, and then the normal Winsock operation can proceed. However, when an event occurs, the completion port is added to the queue by the operating system. The application can then query the core layer to get the completion port.

Here I would like to add some of the above concepts, before explaining [finish] Two words, I would like to briefly mention the two concepts of synchronization and asynchrony, logically speaking to do one after the other to do another thing is synchronization, and at the same time to do two or two or more things are asynchronous. You can also use a single thread and multi-threading to make a metaphor. But we have to distinguish between synchronous and blocked, asynchronous and non-clogging, so-called blocking functions such as the Accept (...), when this function is called, the thread will hang until the operating system notifies it, "Hey brother, someone has come in", the suspended thread will continue to work, and will conform to the " Producer-Consumer "model. Congestion and synchronization look like two points, but they are completely different. Everyone knows I/O device is a relatively slow device, regardless of the printer, modem, or even hard disk, compared to the CPU is extremely slow, sit down and so on I/O completion is a very unwise thing, sometimes the data flow rate is very alarming, Move the data from your file server at Ethernet speed, which can be as high as 1 million bytes per second, if you try to read 100KB from the file server, in the eyes of the user is almost instantaneous, but, to know, your thread executes this command, 10 1 million cycles of CPU time have been wasted. So, we generally use another thread for I/O. Overlapping io[overlapped I/O] is a technology of WIN32, you can ask the operating system to send data to you, and notify you when the transfer is complete. This is the meaning of [done]. This technology allows your program to continue to process transactions while I/O is in progress. In fact, overlapped I/O is done within the operating system with threads. You can get all the benefits of threading without having to pay any painful costs.

The so-called [port] in the completion port is not the port that we mentioned in TCP/IP, so it is completely non-relational. I haven't figured out yet. What is the relationship between an I/O device [I/O devices] and ports [port in IOCP]. It is estimated that this port also confuses a lot of people. IOCP is only used for read and write operations, and file I/O is somewhat similar. Since it is a read-write device, we can only ask for it to be efficient in handling read and write. In the third part of the article you will easily discover the true purpose of IOCP design.


What's the connection between IOCP and the Internet?

int Main ()
{
WSAStartup (Makeword (2, 2), &wsadata);
Listeningsocket = socket (af_inet, sock_stream, 0);
bind (Listeningsocket, (sockaddr*) &serveraddr, sizeof (SERVERADDR));
Listen (Listeningsocket, 5);
int nlistenaddrlen = sizeof (CLIENTADDR);
While (TRUE)
{
Newconnection = Accept (Listeningsocket, (sockaddr*) &clientaddr, &nlistenaddrlen);
HANDLE hthread = CreateThread (NULL, 0, ThreadFunc, (void*) newconnection, 0, &dwtreadid);
CloseHandle (hthread);
}
return 0;
}

Believe that as long as the network of friends, should be familiar with such a structure. After accept, the thread is suspended, waits for a customer to make a request, and then creates a new thread to process the request. When a new thread processes a client request, the initial thread loops back to wait for another customer request. The end of the thread processing after processing the client request.

In the concurrency model described above, a thread was created for each client request. The advantage is that the thread waiting for the request requires little work. Most of the time, the thread is hibernating [because recv is plugged in].

But when the concurrency model is applied on the server side [Windows nt],windows NT Team notes that the performance of these applications is not as high as expected. In particular, handling many simultaneous customer requests means that many threads run concurrently in the system. Because all these threads are operational [not being suspended and waiting for what happens],microsoft realizes that the NT kernel spends too much time translating the context of running threads [context], the threads do not get much CPU time to do their work.

You may also feel that the bottleneck of the parallel model is that it creates a new thread for each customer request. Creating a thread is less expensive than creating a process, but it is far from overhead.

We might as well imagine: if you open n threads in advance, let them jam in that hold[, then you can post all the user's requests to a message queue. The n threads then remove the message from the message queue and process it. You can avoid threads for every user request. Not only reduce the resources of the thread, but also improve the utilization of the thread. It's very good in theory, you want me to wait for a general to come up with a problem, how can Microsoft not consider it?!

The solution to this problem is a kernel object called the I/O completion port, which was first introduced in Windows NT3.5.

In fact, our idea above should be almost the design mechanism of IOCP. In fact, IOCP is not a message queue! How do you say this is related to the word [port]? My understanding is that IOCP is an interface for application and operating system communication.

As for the specific design of IOCP that I can hardly come up, after all, I have not seen the implementation of the code, but you are fully capable of simulation, but performance may ..., if you want to understand IOCP, Jeffrey Ritchter advanced Windows 3rd There is a lot of valuable content in chapters 13th and 14th, and you can take a peek at how the system is doing it all.


Implementation Method

Microsoft provided the corresponding API functions for IOCP, the main two, we look at each:
HANDLE CreateIoCompletionPort (
     HANDLE filehandle,//HANDLE to File
    HANDLE Existingcompletionport,//HANDLE to I/O completion port
    ulong_ptr Completionkey,       //Completion key
    DWORD NumberOfConcurrentThreads//number of threads to execute concurrently
);

Before discussing the parameters, the first thing to note is that the function is actually used for two distinct purposes:
1. Used to create a completion port object
2. Associating a handle [HANDLE] with the completion port

When creating a complete port, we just need to fill in the NumberOfConcurrentThreads parameter. It tells the system the maximum number of threads that are allowed to run concurrently on a completed port. By default, the number of threads is the same as the number of CPUs, but experience gives us a formula:
Number of threads = number of CPUs * 2 + 2
To make the completion port useful, you must associate it with one or more devices. This is also done by calling CreateIoCompletionPort. You're going to pass a handle to the function that has an existing completion port, and since we're dealing with network events, it's going to pass the client's socket as a handle. and a completion key [a 32-bit value that means something to you, which is a pointer, the operating system doesn't care what you preach]. Whenever you associate a device to a port, the system adds an information record to the list of devices for that completion port.

Another API is
BOOL GetQueuedCompletionStatus (
    HANDLE Completionport,       //Handle to completion port
    Lpdword Lpnumberofbytes,      //Bytes Transferred
     pulong_ptr lpCompletionKey,//File completion key
     lpoverlapped *lpoverlapped,//Buffer
    DWORD dwmilliseconds    Optional timeout value
);

The first parameter indicates which completion port the thread will monitor. Many service applications simply use an I/O completion port, and all subsequent notifications of I/O requests are sent to that port. Simply put, getqueuedcompletionstatus causes the calling thread to hang until an entry occurs in the I/O completion queue for the specified port or until it expires. The 3rd data structure associated with the I/O completion port is the information in the I/O entry that the thread gets completed: The number of bytes transferred, the address of the completion key and the overlapped structure. This information is returned to the thread by the Lpdwnumberofbytestransferred,lpdwcompletionkey and lpoverlapped parameters passed to Getqueuedcompletionsatatus.

Based on what has been said so far, first build a frame. The following shows you how to use the completion port to develop an echo server. Roughly as follows:
1. Initialize Winsock
2. Create a completion port
3. Create a fixed number of threads based on the number of server threads
4. Prepare a socket for bind and listen
5. Enter the loop accept waiting for customer request
6. Create a data structure to accommodate sockets and other related information
7. Associate the connected socket with the completion port
8. Deliver a ready-to-accept request
Repeat the process from 5 to 8.
Well, let's show the details of the operation with the specific code.

[Go] understanding I/O completion Port