I/O multiplexing Select

1. What is I/O multiplexing

about what is I/O multiplexing, there is a good answer on the understanding, you can refer to the answer of Lo Zhiyu predecessors.

Here's a record of your understanding. I think to understand this term has to go from two aspects, one is: what is the concept of multi-way? The second is: what is the reusable thing? Let's start with the first question. Multi-path refers to a number of independent I/O flows, I/O flow can understand that: Read is a stream (called a read stream, such as an input stream), write is a stream (called the write stream, such as the output stream), the exception is a stream (called the exception stream), each stream with a file descriptor, The same file descriptor can represent both read and write streams. Look at the second aspect, what is the reuse? Multiplexing is a thread that uses a thread to track the status of each IO, and then it can process all of the IO with one of the threads.

Of course, do not mention what I/O multiplexing can also be in a thread to process all the IO stream, with a while loop one after another to solve it? Then why do you propose this technique? The reason is that the method we have just thought (polling) is inefficient and resource utilization is not high. Imagine that if an IO is set to block Io, then the other IO will be stuck and the other IO resources would be wasted. On the other hand, assuming that all IO is set to non-blocking, the CPU does not have to do anything else all day long, in this constantly asked, now you can do IO operation, until there is a device ready for the environment to do IO, that is, in the device preparation IO environment this time, the CPU is not necessary to blindly ask, There was no result in asking.

Then the hardware developed, with a multi-core concept, there is a multi-threaded. This time can be done, to an IO I open a thread, so that no longer polling. However, managing threads is a waste of system resources, and programmers are beginning to have headaches, and the interaction between threads is cumbersome. As the complexity of the program goes up and up, IO efficiency is likely to improve, but the efficiency of software development may be reduced.

So there is the technique of I/O multiplexing. In short, it is a thread that tracks multiple IO streams (read, write, exception), but does not use polling, but the device itself tells the program which stream is available, freeing up the CPU, and taking full advantage of IO resources, the following is the main way to implement this technology, Linux under this technology has three implementations, Select,poll,epoll. Today, we mainly record my understanding of select, from interface to principle to realization.

2.Select interface

#include <sys/select.h>

int Select (int Nfds, fd_set *readfds, fd_Set *writefds, fd_set *exceptfds, struct Timeval *timeout);

• Readfds, read stream collection, which is what programmers want to read from these descriptors
• Writefds, the write stream collection, which is what programmers want to write to these descriptors
• Exceptfds, the exception flow set, which is the intermediate process sends an exception
• Nfds, the top three events, the largest file descriptor +1
• Timeout, the programmer's tolerance, the time to wait

struct timeval{

Long Tv_sec;//second

Long Tv_usec;//minisecond

}

Timeout has three kinds of values:

• Null,select has been blocked, knowing that at least one file descriptor in the Readfds, Writefds, Exceptfds collection is available to wake up
• 0,select does not block
• Timeout_value,select blocked in Timeout_value this time period

If it has to be associated with the word "multi-path", it is the number of Readfds+writefds+exceptfds and the way it is.

In addition, there is a set of operations related to FD_Set

• Fd_set (fd, _fdset), adding FD to the _fdset set
• FD_CLR (fd, _fdset), clears the FD from the _fdset collection
• Fd_isset (fd, _fdset) to determine if FD is in the _fdset set
• Fd_zero (_Fdset), clear _fdset with descriptor

3. Select Realization principle

The implementation of select depends on the device's drive function Poll,poll's function is to check which stream of the device is available (a device typically has three streams, read stream, write stream, abnormal flow of equipment exception), if one of the streams is available, returns a mask (representing the available stream), if not available, Adds the current process to the stream waiting queue for the device, such as a read-wait queue, a write-wait queue, and returns a resource that is not available.

The select takes advantage of the poll feature, first letting the programmer know which IO streams they care about (denoted by file descriptors, Readfds, Writefds, and Exceptfds above), And let the programmer tell themselves the scope of these streams (that is, the Nfds parameter above) and the program tolerance (timeout parameter), then select will copy them to the kernel, in the kernel to call the stream corresponding to the device driver poll function, When all the flow in the range is the descriptor, he checks to see if at least one stream has occurred, modifies the three stream sets, empties them, and then joins the resulting stream into the corresponding collection, and select returns. If not, sleep, give up the CPU, until a certain stream of a device is available, to wake up the process blocking the flow, this time, the process of invoking select restarts the traversal of all the descriptors in the range.

Looking directly at this step may be a good idea.

• 1. Copy Nfds, Readfds, Writefds and Exceptfds to the kernel
• 2. Traverse each stream within the [0,nfds] range, and call the drive poll function of the device corresponding to the stream.
• 3, check whether there is a flow occurs, if there is a occurrence, the stream set the corresponding category, and execute 4, if no flow occurs, perform 5. Or timeout=0, perform 4
• 4. Select returns
• 5. Select blocks the current process, waits for the device to be awakened by the stream, and executes 2 when it is awakened. Or timeout expires, execute 4

Then add a copy of the select in the kernel flowchart

　　　　　　

4.Select Implementation

The core implementation of select is Do_select, so take a look at the Do_select source code, non-complete source, only the key parts are retained

intDo_select (intN, fd_set_bits *fds, S64 *Timeout) {retval=0;//retval is used to save the number of descriptors that are already prepared, starting with 0          for (;;) {unsignedLong*RINP, *ROUTP, *rexp, *INP, *OUTP, *exp; Long__timeout;    Set_current_state (task_interruptible); //changes the current process state to task_interruptible and can be interruptedINP= fds->inch; OUTP = fds-> out; Exp = fds->ex; RINP= fds->res_in; ROUTP = fds->res_out; Rexp = fds->res_ex;  for(i =0; I < n; ++RINP, ++ROUTP, ++rexp) {//iterate through each descriptorUnsignedLong inch, out, ex, all_bits, bit =1, Mask, J; unsignedLongRes_in =0, Res_out =0, RES_EX =0; Const structFile_operations *f_op =NULL; structFile *file =NULL; inch= *inp++; out= *outp++; ex = *exp++; All_bits=inch| out|ex; if(All_bits = =0) {i+ = __nfdbits;the type of//all_bits is unsigned long int, the size is 4 bytes 32 bits, All_bits=0, which indicates that 32 descriptors (streams) are not in Readdfs, Writedfs, Execptdfs collections, so i+=32, and __nfdbits=32.                                      Continue; }                              for(j =0; J < __nfdbits; ++j, ++i, bit <<=1) {//traverse each bit in each long word                                     intfput_needed; if(I >=N) Break; if(! (Bit &all_bits)) Continue; File= Fget_light (i, &fput_needed); if(file) {F_op= file->F_op; MARK (Fs_select,"%d%lld", I, (Long Long)*timeout); Mask=Default_pollmask; if(F_op && f_op->poll)Mask= (*f_op->poll) (file, retval?)null:wait);//Call the device's driver poll function fput_light (file, fput_needed); if(Mask & Pollin_set) && (inch&bit)) {res_in|= bit;//If this descriptor is readable, place this position bitretval++;//returns the number of descriptors plus 1                                               }                                               if(Mask & Pollout_set) && ( out&bit)) {Res_out|=bit; retval++; }                                               if(Mask & Pollex_set) && (ex &bit)) {RES_EX|=bit; retval++; }                                     }                            }                            if(res_in)*RINP =res_in; if(res_out)*ROUTP =res_out; if(RES_EX)*rexp =res_ex; } Wait=NULL;                   if(retval | |!*timeout | |signal_pending (current)//if retval!=0, that is, there are Readdfs, Writedfs, Execptdfs at least one occurrence, jump out of the loop Break;/ * Handle the timeout parameter below */
　　　　　　　　　　__timeout=schedule_timeout (__timeout); if(*timeout >=0)                            *timeout + =__timeout; } __set_current_state (task_running);
returnretval;}

Resources:

Analysis of the implementation principle of select function
Wait Queue (ii)

I/O multiplexing Select