Performance indicators of asynchronous Io, APC, Io completion port, thread pool, and high-performance server (5)

Server performance indicators

As a network server program, performance is always the first indicator. Performance can be defined as the number of tasks that can be processed in a given hardware condition and time. The server design that maximizes the use of hardware performance is a good design.
A well-designed server should also consider the average service. For each client, the server should provide the average service to each client, so that a client cannot be served for a long time and the server is "Hungry.
Scalability. That is to say, as the hardware capability increases, the server performance will grow linearly.

High Performance

The computing of an actual server is very complicated, often mixed with I/O computing and CPU computing. Io computing refers to the computing model in which I/O is the main component, such as the file server and mail server. It combines a large number of network I/O and file I/O; CPU computing refers to the absence or absence of IO in computing tasks, such as encryption/decryption, encoding/decoding, and mathematical computing.
In CPU computing, the single-line and multi-thread models have the same effect. "In a single processor computer, the concurrent execution speed of CPU-based tasks cannot be faster than that of serial execution, but we can see that, the extra overhead for Windows NT offline creation and switching is very small; for very short computing, concurrent execution is only 10% slower than serial execution, and with the increase of the computing length, these two times are very close."
It can be seen that for pure CPU computing, the multi-threaded model is not suitable if there is only one CPU. Consider a service that executes intensive CPU computing. If there are dozens of such threads for concurrent execution, too frequent Task Switching leads to unnecessary performance loss.
In programming implementation, the single-thread model computing model is inconvenient for Server programming. Therefore, it is more appropriate to use the thread pool working model for CPU computing. The queueuserworkitem function is very suitable for placing a CPU computing into a thread pool. Thread Pool implementation will try to reduce unnecessary thread switching and control the number of concurrent threads to the number of CPUs.
What we really need to care about is Io computing. Generally, network server programs are accompanied by a large amount of Io computing. The way to improve performance is to avoid waiting for the end of Io, resulting in idle CPU. Try to use hardware capabilities to allow one or more Io devices to run concurrently with the CPU. The asynchronous Io, APC, and Io Completion Ports described above can achieve this goal.
For network servers, if the number of concurrent requests on the client is relatively small, a simple multi-threaded model can be used. If a thread is suspended because it waits for the completion of the I/O operation, the operating system will schedule another ready thread to run, resulting in concurrent execution. Classic Network Server logic mostly adopts multi-thread/multi-process mode. When a client initiates a connection to the server, the server will create a thread for the new thread to process subsequent transactions. This programming method that represents a client object with a special thread/process is very intuitive and easy to understand.
For large network server programs, this method has limitations. First, the cost of creating a thread/process and destroying a thread/process is very high, especially when the server uses TCP "short connection" or UDP communication, for example, in HTTP, the client initiates a connection and sends a request. After the server responds to the request, the connection is closed. If the HTTP server is designed in the Classic mode, too frequent thread creation/destruction will have a bad impact on the performance.
Second, even if a TCP "persistent connection" is adopted in a protocol, the client keeps this connection after connecting to the server. The classic design method also has disadvantages. If the client has a high number of concurrent requests and many clients wait for the server to respond at the same time, too many threads will be executed concurrently, and frequent thread switching will use part of the computing power. In fact, if the number of concurrent threads is too large, physical memory is often used up too early, and most of the time is spent on thread switching, because thread switching will also cause memory paging. Eventually, the server performance drops sharply,
The thread pool is the only solution for a network server that needs to handle concurrent requests from a large number of clients at the same time. The thread pool can not only avoid frequent thread creation and thread destruction, but also process a large number of concurrent client requests with a small number of threads.
It is worth noting that we do not recommend any of the above techniques for designing a low-pressure network server program. It is unwise and stupid to make things complicated when a simple design can complete the task.