Epoll Learning: A high-performance server processing framework

Finally, I started to learn epoll. Although there are still many things I don't understand, I believe everything will be clearer after I write a specific framework on my own from theory to practice.

1. First, a memory pool is required:
· Reduce frequent allocation and release, improve performance, and avoid memory fragmentation issues;
· The ability to store variable-length data should not be foolish enough to pre-allocate only one maximum length;
· Implementing the Memory Pool Based on the slab algorithm is a good idea: allocate multiple blocks of different sizes and return the smallest block greater than the length of the request. For containers, processing the allocation and recovery of fixed blocks is quite easy to implement. Of course, remember to design it as thread-safe. Spin locks are better, and read/write spin locks are better.
· Managing the growth of allocated content is a problem. For example, the first time a 1 kb space is required, 4 kb space is required for the second time as data is continuously written. The expansion space is easy to implement, but the expansion involves data copying. Even if the demand for expansion is huge and hundreds of megabytes of data are needed, it is hard to handle it. There is no better idea for the moment. Just like STL, although it is inevitable to copy data in an exponential growth allocation strategy, the probability of redistribution is at least getting lower and lower.
· As mentioned above, it is a good way to use memory ing files for management if it is needed to expand data of hundreds of megabytes: After the ing file, although it occupies a large amount of virtual memory, however, the physical memory is allocated only when it is written. With madvice () and sequential write optimization recommendations, the physical memory consumption will also decrease.
· It is unwise to use string or vector to manage memory. Although it is very simple, STL is not suitable for server software development, especially when the stability and performance requirements are high.

2. The second thing to consider is the object pool, which is similar to the memory pool:
· Reduce the allocation and release of objects. In fact, the C ++ object is struct. It is not difficult to manually initialize and clean the constructor from the constructor, so as to keep the circular utilization of the same buffer zone.
· An object pool can be designed to store only one object. The implementation of the Object pool is actually the Pooled management of fixed memory blocks, which is very simple. After all, the number of objects is very limited.

3. The third requirement is queue:
· If we can predict the processing capability of the limit, it would be good to use a fixed-size ring queue as a buffer zone. A producer and a consumer are common application scenarios. The circular queue has its classic "lock-independent" algorithm. In a scenario where a thread reads and writes data from a thread, the implementation is simple and the performance is high, resource allocation and release are not involved yet. Okay, it's really good!
· When multiple producer consumers are involved, TBB: concurent_queue is a good choice, thread security and concurrency are good, that is, it does not know whether resource allocation and release are well managed.

4. The fourth one requires a ing table or a hash table:
· Because epoll is triggered by events, and a series of processes may be dispersed among multiple events, you must keep the intermediate state so that when the next event is triggered, you can continue processing at the last processing location. To be simple, the hash_map of STL is okay, but you have to handle the lock yourself. It is very troublesome to use it in a multi-threaded environment.
· For hash tables in multi-threaded environments, TBB: concurent_hash_map is the best.

5. The core thread is the event thread:
· The Event thread is the thread that calls epoll_wait () to wait for the event. In the example code, when a thread does everything and needs to develop a high-performance server, the event thread should focus on processing the event itself, place the socket handle of the trigger event in the corresponding processing queue, and the specific processing thread is responsible for the specific work.

6. Accept () is a separate thread:
· The socket handle of the server (that is, the interface that calls BIND () and listen () is best used to perform accept () in a separate thread. It doesn't matter whether it is blocked or not blocked, compared with the communication across the server, user access is only a small part. In addition, accept () is not placed in the loop of the event thread, reducing the judgment.

7. The receiving thread is separate:
· The receiving thread extracts the socket handle from the queue where an epollin event occurs, and then calls Recv to receive data on the handle until there is no data in the buffer zone. The received data is written into the hash table with socket keys. The hash table has a self-increasing buffer and stores the data sent from the client.
· This processing method is suitable for applications with a small amount of data sent from the client, such as HTTP servers. If it is a file upload server, the receiving thread will always process the massive data of a connection, data Processing on other clients is getting hungry. Therefore, this design is not applicable to scenarios such as file upload servers.

8. A separate sending thread:
· The sending thread obtains the socket handle of the data to be sent from the sending queue and calls send () to send the data to the client. The socket handle is saved in the queue. You need to use the socket handle in the hash table to locate the specific object. As mentioned above, the client information object not only has a longer receiving data buffer, but also a longer sending data buffer. The specific working thread does not directly call the send () function when sending data, but writes the data to the sending data buffer, and then places the socket handle to the sending thread queue.
· Another situation where the socket handle is placed in the sending thread queue is that the epollout event occurs in the event thread, which indicates that the TCP sending buffer has available space, at this time, you can put the socket handle in the sending thread queue and trigger the send () call;
· It should be noted that when the sending thread sends a large amount of data, it will not be able to send messages when sending () is frequently called until the TCP sending buffer is full. In this case, if you wait cyclically, the sending work of other users will be affected. If you do not continue sending, The epoll et mode may no longer produce events. The solution to this problem is to create a queue in the sending thread, or set a flag on the user information object, and wait until the thread is idle to continue sending the unsent data.

9. A timer thread is required:
· One expert who uses epoll said: "It is almost impossible to rely solely on epoll to manage descriptors without disclosing them. The complete solution is simple, that is, to set the timeout time for each FD. If the time exceeds the timeout time, the FD will be closed if it is not active ".
· Therefore, the timer thread regularly trains the entire hash table and checks whether the socket is not active within the specified time. Inactive sockets are considered timeout, and the server closes the handle and recycles resources.

10. Multiple worker threads:
· The worker thread is triggered by the receiving thread: each time the receiving thread receives data, it puts the socket handle with data into a work queue. The worker thread then obtains the socket handle from the work queue, query the hash table, locate the user information object, and process the business logic.
· If a worker thread needs to send data, it first writes the data to the sending buffer of the user information object, and then places the socket handle in the sending thread queue.
· For the task queue, the receiving thread is the producer, multiple working threads are the consumer, multiple working threads are the producer for the sending thread queue, and the sending thread is the consumer. Pay attention to the lock issue here. If TBB: concurrent_queue is used, it will be much easier.

11. It is not enough to use only the scoket handle as the key of the hash table:
· Assume that the event thread has just put a socket into the receiving queue due to an epollin event, but the client is disconnected due to an exception, the event thread deletes this item in the hash table because of the epollerr event. Assume that the receiving queue is long and the exception socket is still in the queue. When the receiving thread processes the socket, it cannot index the objects in the hash table through the socket handle.
· If the index is not available, the difficulty is that the socket handle is immediately used by another client. The access thread creates an object in the hash table for scoket. At this time, the two sockets with the same handle are actually two different clients. In extreme cases, this situation is possible.
· The solution is to use socket FD + sequence as the key of the hash table, and sequence is obtained by the Access thread accumulating an integer value after each accept. In this way, even if the socket handle is reused, no problem will occur.

12. Monitoring, consider the following:
· The most common problem in the framework is the working thread: when the processing speed of the working thread is too slow, various queues will soar and the server will eventually crash. Therefore, you must limit the maximum size allowed by each queue and monitor the processing time of each worker thread. If this time is exceeded, you should end the working thread in a certain way.