Linux multi-thread socket programming experience

Linux multi-thread socket programming experience-general Linux technology-Linux programming and kernel information. The following is a detailed description. From: lingxiao web tribe

Some time ago, a new web model was built on linux, and epoll was used to replace IOCP. It was tested that the performance was improved a lot and the throughput was also improved, network Programming in linux is not just plain-spoken, and multithreading is even more troublesome. However, the essence of the epoll model is event-driven, this model provides the ability to keep the connection socket increasing in a straight line and the performance will not decrease in a straight line. Throughout epoll kueuen select and so on, all of them are solving the problem that a socket does not need a thread, separate events.

Some experiences have been found in ningx (someone used it to maintain 30 thousand processing connections at the same time). These are not only servers that use models such as web, it is also suitable for all others, such as games and ftp servers.

The key to maintaining a high number of online users is that the event processing model requires m: n, m far
Nginx strictly separates the processing programs (this is why the Flower web server I am working on is also based on this principle). When epoll and other events are notified, nginx maintains two linked lists globally, A read connection table, a write linked list, epoll, and other system network notifications notify a socket that the status is saved in the corresponding variable after it is operable, and then the m threads are used to poll and execute the task, in this way, the thread does not have to wait for network io congestion. A normal read socket data needs to be cyclically until the data is read or an error is generated, I maintain a unified socket fd table globally, which is determined by the linux features, because the linux socket fd starts from a very small value, in addition, there will be no duplicates within the same time (it is estimated that there is a thread lock inside). All fd itself is an important identifier, and a large fd will be reused from an early stage after it is released, winsows is different. It is a long pointer. All my global status tables need a logo that uses fd as the index. It is a general number that maintains the number of online users at the same time. I set a 20000 status table. The memory for the current stack is very small and will not be used up.

Then, I set two independent State linked lists for read and write, which are processed by m threads. The difference is that the two tables have only one fd entry, the rest of the data is saved in the global state table of heap allocation. I set many Status values for the global state table, such as read write readed writed needRepetRead needrepetWrite restart, in this way, I can use the new State linked list to maintain other processes. The status of the global table is different when the locks are different, thus achieving the collaboration of multiple processing programs, the mutex lock only exists in each processing link. If the read link is slow, the number of read threads can be increased. If the write speed is slow, the write speed can be increased, if the processing process is slow, you can increase the processing speed to facilitate later debugging and optimization of the system. However, the processing of Global tables is not affected in every step, it will not cause global status to fail because a thread is blocked or dead.

In the case that the initial model adopted sendfile + epoll, I tested that my server has slightly exceeded nginx. The following section further standardizes and optimizes the expansion.