Linux -- I/O multiplexing

 

Remember Master Steven S.

Best references:

1. Master from the Internet.

2. UNP V1 Chapter 1.

3. Man select, man pselect, man epoll, and man poll

Article 1: Overview

Reuse is a great concept! What is I/O multiplexing (I/O multiplexing? Specifically, when the program you write needs to process multiple numbers simultaneously (socket, file, or device), you do not know when to do so (for example, you can read data) which number to operate (read/write. At this time, I/O multiplexing is required.

 

Unpv1 provides a definition. I/O reuse is a kind of kernel capability that allows the process to "Warn" in advance, so that once the kernel discovers that the process is notified in advance, one or more I/O conditions (that is, descriptors) are specified) when the system is ready (read/write is available), the kernel notifies the process. Linux has four calls that enable I/O multiplexing: select and poll inherit from UNIX systems. Pselect is from select to POSIX. Epoll is unique to linux2.6 kernel.

Article 2: I/O model

(0) generally, the input operation usually contains the following two stages. Please note:

1. Wait for the data to be ready.

2. Copy data from the kernel to the process.

(0.1) For example, the input operation on the socket:

1. Data arrives from the network. When the data arrives, it is collected to the corresponding buffer zone in the kernel.

2. Copy data from the kernel buffer to the process port buffer.

(1) unpv1 summarizes the Unix I/O model:

1. Blocking I/O: the process is blocked by the I/O system call (read, write, sendto, recvfrom, etc.) until the I/O conditions are ready (readable/write or abnormal) after processing, the process is returned from the I/O system call.

2. Non-blocking I/O. That is, the non-blocking I/O system call is called in polling mode.

 

3. I/O multiplexing.

4. multi-thread blocking I/O. Allocate the number of records to be processed to multiple threads, and each thread independently calls the I/O system call for independent processing.

 

5. Signal-driven I/O. Similar to I/O multiplexing, when the I/O condition is ready, the signal notification process is used to call the I/O system call processing.

 

6. asynchronous I/O. Aio_read is used to tell the kernel how to process the data. After the kernel is processed, a signal is sent to us. We can also notify us after processing the data in other ways.

Note: whether the I/O system calls are blocked depends only on whether the number of the operations they perform is blocked. Fcntl can be set.

(2) These six I/O models are divided into two categories:

Synchronous I/O model: 1st ~ 5 I/O model. After all, I/O calls must be called by the user process and blocked.

Asynchronous I/O models: 6th I/O models. I/O calls are executed by the kernel and the process is not blocked.

 

POSIX terms for Synchronous I/O (synchronous I/O operations, asynchronous I/O) operations are defined as follows:

 

Synchronous I/O operation: the request process is blocked until the I/O operation is completed.

 

Asynchronous I/O operations: do not cause process blocking.

Note: to identify whether an I/O operation is asynchronous, you only need to check who performed the I/O operation.

Article 3: pselect and select

# Include <sys/select. h>

(0) int pselect (INT NFDs, fd_set * restrict readfds, fd_set * restrict writefds, fd_set * restrict errorfds,

Const struct timespec * restrict timeout, const sigset_t * restrict sigmask); // select POSIX version.

 

 

Int select (INT NFDs, fd_set * restrict readfds, fd_set * restrict writefds, fd_set * restrict errorfds,

 

Struct timeval * restrict timeout );

(0.1) parameter NFDs: specifies the number of descriptors to be tested (not the maximum value), because the descriptor starts from 0, so his value is the maximum descriptor to be tested plus 1 (one more 0 ).

(0.2) readfds, writefds, and errorfds: the three parameters are of the fd_set type. These three parameters have two purposes in different time periods of function calls:

(0.2.1) passed in as a parameter: We put the descriptors of interest into the event type according to the event type of interest respectively. How to put it? See.

(0.2.2) return value when the function is returned: these three parameters are loaded with descriptors that we care about and are ready for the I/O condition, and the incomplete description is clear. Therefore, it is very important to reset these three parameters each time you call select.

(0.3) parameter Timeout: Type in select and pselect. The supertime types are as follows:

/* Posix.1b structure for a time value. This is like a 'struct timeval' but has nanoseconds instead of microseconds .*/

Struct timespec {// added during use
-D _ need_timespec

_ Time_t TV _sec;/* seconds .*/

Long int TV _nsec;/* nanoseconds. */nanosecond.

};

/* A time value that is accurate to the nearest microsecond but also has a range of years .*/

Struct timeval {

_ Time_t TV _sec;/* seconds .*/

_ Suseconds_t TV _usec;/* microseconds. */subtle.

};

(0.4) parameter sigmask: Signal mask.

 

 

 

 

(1) descriptor FD and descriptor set fdset operation macro:

Void fd_clr (int fd, fd_set * fdset); // congfdset deletes FD.

Int fd_isset (int fd, fd_set * fdset); // test whether the FD is in fdset.

Void fd_set (int fd, fd_set * fdset); // Add FD to fdset

Void fd_zero (fd_set * fdset); // reset, similar to bzero.

(2) Select fd_setsize limit. The number of descriptors that can be opened in Linux is limited only by memory and management. However, at the beginning of the select function design, the internal implementation of select references fd_setsize, that is, the maximum number of descriptors that can be processed by select is fd_setsize, generally 1024. /* Number of descriptors that can fit in an 'fd _ set'. */# DEFINE _ fd_setsize1024

 

Article 4: poll

# Include <poll. h>

(0) int poll (struct pollfd FDS [], nfds_t NFDs, int timeout); // epoll is an enhanced version of poll

(0.1) the FDS parameter is an array pointing to struct pollfd. He is the core of poll.

Struct pollfd {/* Data Structure describing a polling request. * // each descriptor corresponds to a struct pollfd structure.

Int FD;/* file descriptor to poll. * // the descriptor we care about.

Short int events;/* types of events poller cares about. * // events that the descriptor cares about

Short int revents;/* types of events that actually occurred. * // The time when the descriptor FD occurs when poll returns.

};

The values of member events and revents are as follows:

/* Event types that can be polled. these bits may be set in 'events' to indicate the interesting event types; they will appear in 'revents' to indicate the status of the file descriptor. */

# Define pollin0x001/* There is data to read. * // read

# Define pollpri0x002/* There is urgent data to read. * // out-of-band data arrival

# Define pollout0x004/* Writing now will not block. */writable

 

# If defined _ use_xopen | defined _ use_xopen2k8/* these values are defined in xpg4.2. * // rarely used

# Define pollrdnorm0x040/* normal data may be read. * // read General Data

# Define pollrdband0x080/* priority data may be read. * // The priority must be readable.

# Define pollwrnorm0x100/* Writing now will not block. * // read General Data

# Define pollwrband0x200/* priority data may be written. * // The priority data can be written.

# Endif

 

# Ifdef _ use_gnu/* These are extensions for Linux .*/

# Define pollmsg0x400

# Define pollremove0x1000

# Define pollrdhup0x2000

# Endif

 

/* Event types always implicitly polled for. These bits need not be set in 'events', but they will appear in 'revents' to indicate the status of the file descriptor .*/

# Define pollerr0x008/* error condition. */Error

# Define pollhup0x010/* hung up. */suspended

# Define pollnval0x020/* invalid polling request. * // invalid Descriptor

(0.2) parameter NFDs: indicates the size of the array.

(0.3) parameter Timeout: timeout. The value is as follows, in milliseconds:

Inftim: Always wait. Linux does not define this value. You can manually define it as-1.

0: return immediately without blocking the process.

> 0: Wait for the specified number of milliseconds.

Article 5: FAQ

1. epoll is an enhanced version of poll.

 

 

2. poll does not have the upper limit of the select descriptor. The maximum number of descriptors is only in the memory size structure, that is, the number of struct pollfd in the memory.