After the iocp Server framework (including the example of calling Source Code) is re-designed, it is hard to believe in Performance Improvement and stability. Let's talk about my design ideas!

I. Design Ideas
1. design model;
The I/O receiving and receiving threads and employ task threads must be separated. Otherwise, if I/O threads are only taken into account to handle all things, once a service interface runs slowly, it will inevitably cause IO network congestion, so what is the use of this background service?
The employ task thread assignment algorithm depends on the specific service. If the data transmission volume of all services is about the same size and the response time is about the same, you can use the average order allocation method, otherwise, one or more employ task threads can be assigned for processing based on specific business calls.
If you do better, you should monitor each employ task thread. For example, you can determine which thread to allocate based on the length of the existing task queue.
2. Performance Assurance;
During normal communication, there must be basically no shared data lock mechanism between connections and threads; otherwise, the performance will be compromised;
The buffer access mechanism should adopt a circular buffer mechanism similar to the Linux kernel technology. Its biggest characteristic is that if there is only one write thread and a read thread at the same time, this circular buffer does not need any mutex lock mechanism, this ensures the performance;
Some parameter optimization for the socket itself, such as setting so_sndbuf to zero, can reduce the CPU time consumed for user-to-kernel replication.
3. handle errors;
One of the biggest difficulties of iocp is to recycle resources when the connection is disconnected. We recommend two methods:
3.1 adopt a latency recovery policy. after ensuring that all buffer objects return to the queue within a certain period of time, release all the resources of the connection. The premise is that your service interface is running at a high speed, generally, about 30 seconds is enough ?!
3.2 adopt an immediate recovery policy, but you must consider adding a buffer counter to the multi-thread scenario. After all the buffers are collected to the current connection object, when the number of pending connection tasks in the employ task thread is 0, the task will be recycled immediately.
4. Fault Tolerance Mechanism;
It mainly checks whether the packet itself is illegal. Generally, it adopts the Protocol mode of 4-byte header length bit + N-byte data bit, so that the server only needs to interpret the packet header each time to determine whether the packet is valid,
The client API must ensure the correctness of the packets, and avoid sending the packets to the server in a sort order;
If the server receives an illegal data packet from a connection, it should take the initiative to close the connection.
5. Business access.
The business inheritance development interface generally only needs to pay attention to three methods: connection, disconnection, and data reading;
The business interface only needs to inherit and reload the Data Reading method of the parent class, and call the sending method of the parent class after constructing the reply data;
It is necessary to ensure the efficiency of the business interface implementation, so as to avoid the occurrence of a pot of fish.

Ii. Test
Recommended Performance Testing example:
1. Each client sends an average of 0.5 requests per second. The packet size is about 100 bytes;
2. The server replies 1024 bytes;
3. There are 10000 connections;
4. The server is r710 and 8 cores;
5. The network is Gigabit.

The mean value of the test response should be no more than 500 subtle, and a good response may be less than 100 subtle, otherwise your design will certainly have problems.

Iii. Use of mutex lock
Your server design generally has only two lock mechanisms:
1. Each connection object is self-locked to keep the synchronization of some variables in the connection object;
2. Set of all connections, generally map, used for insert and remove client connections.

In addition, you should not add other locks.