High Performance Server Development Basic Series (i) Division of the main thread and the worker thread

Server side in order to be able to smooth processing of multiple client links, generally in a Thread AAccept the new client connection and generate a new connection for the socket FD, and then add these newly connected SOCKETFDFor several additional worker threads B1, B2, B3, B4, these worker threads handle the network IO events on these new connections (that is, sending and receiving data), while also processing other transactions in the system. Here we refer to thread A as the main thread, B1, B2, B3, B4, etc. called worker threads. The code framework for worker threads is generally as follows:

while (!m_bQuit)  {    epoll_or_select_func();    handle_io_events();    handle_other_things();}

The IO event on the socket FD is detected by Select () or poll/epoll () in epoll_or_select_func () , and the next step if there are these events handle _io_events () Handles these events (sending and receiving data), and may do some other system tasks after they are done, called handle_other_things ().

There are three benefits to doing so:

Thread A only needs to handle the arrival of the new connection, without handling the network IO event. Because the network IO event processing is generally relatively slow, if the thread a inside both processing new connections and processing network IO, it is possible that the threads are busy processing IO events, and can not handle the client's new connections in a timely manner, this is not good.

New connections received by thread A can assign new socket FD to worker threads based on a certain load balancing principle. Commonly used algorithms, such as round robin, that is, the polling mechanism, that is, assuming that the connection is not considered in the middle of the situation, a new connection to the allocation to B1, and another to assign to B2, then a distribution to B3, and then a distribution to B4. So repeatedly, that is, thread A records the number of socket FD on each worker thread, which maximizes the balance of resources, avoids the "busy" work threads, and the "idle" behavior of some worker threads.

Even if the worker thread is not full, you can let the worker thread do other things. For example, there are now four worker threads, but there are only three connections. Then thread B4 can do something else in handle_other_thing () .

A very important efficiency issue is discussed below:

In the while loop, functions such as epoll_wait/poll/select inepoll_or_selec_func () generally set a time-out. If the set time-out is 0, those worker threads will actually be idling and wasting CPU time slices without any network IO time and other task processing. If the set timeout time is greater than 0, in the absence of network IO time, Epoll_wait/poll/select still has to suspend the specified time to return, causing handle_other_thing () not to execute in time, affect other tasks can not be dealt with in a timely manner, that is to say that other tasks once produced, its processing has a certain delay. That's not good either.

So how do we solve the problem?

In fact, the effect we want to achieve is that if there is no network IO time and other tasks to deal with, then these worker threads are best to hang up instead of idling, and if there are other tasks to process, these workers will be able to handle these tasks immediately instead of epoll_wait/poll/ Selec start processing These tasks only after the specified time has been suspended.

We take the following approach to solve the problem, in the case of Linux, regardless of whether there is a file descriptor fd on EPOLL_FD, we bind it to a default FD, which is called Wake FD. When we need to handle other tasks, we write 1 bytes to this wakeup FD, so that the FD immediately becomes readable, and theepoll_wait ()/poll ()/select ( ) function is immediately awakened and returned, The next step is to execute handle_other_thing () and other tasks are handled. Conversely, when there are no other tasks or network IO events, Epoll_or_select_func () hangs there and does nothing.

This wakeup FD, on the Linux platform can be implemented in the following ways:

Pipeline pipe, create a pipe and bind the pipe to the EPOLL_FD. When needed, a byte is written to the end of the pipe, and the worker thread is immediately awakened.

New EVENTFD for Linux 2.6:

The same step is to bind the generated eventfd to the EPOLL_FD. When needed, writes a byte to the EVENTFD, and the worker thread is immediately awakened.

The third method is the most convenient. Linux-specific Socketpair,socketpair is a pair of interconnected sockets, equivalent to the server side and the client's two endpoints, each end can read and write data.

int socketpair(int domain, int type, int protocol, int sv[2]);

Call this function to return the two socket handle is sv[0], and sv[1], in one of the write bytes, in another to receive bytes.

Binds the socket of the received byte to the EPOLL_FD. When needed, writes a byte to another write socket, and the worker thread is immediately awakened.

If you are using Socketpair, then the domain parameter must be set to Afx_unix.

Because the Windows,select function only supports the detection of a socket, this kind of fd, so Windows generally only use the principle of Method 3. And you need to manually create two sockets, then one to connect to the other, to bind the read section to the FD of select. This is where you need to be aware when writing across two platform code.

Welcome to the public number "Easyserverdev". If there is any technical or professional problems need me to help, can through this public number to contact me, this public number not only share high-performance server development experience and story, but also free for the vast number of technical friends to provide technical questions and professional doubts, you have any questions can be directly in the public message, I will reply to you as soon as possible.

High Performance Server Development Basic Series (i) Division of the main thread and the worker thread