Epoll introduction and Examples

After man epoll, the following results are obtained:

Name
Epoll-I/O Event Notification facility

Synopsis
# Include <sys/epoll. h>

Descr resume ption
Epoll is a variant of Poll (2) that can be used either as edge or level
Triggered interface and scales well to large numbers of watched FDS.
Three System CILS are provided to set up and control an epoll set:
Epoll_create (2), epoll_ctl (2), epoll_wait (2 ).

An epoll set is connected to a file descr extends ptor created by epoll_create (2). Interest for certain file descr extends ptors is then registered
Epoll_ctl (2). Finally, the actual wait is started by epoll_wait (2 ).

In fact, all explanations are redundant. According to my current understanding, the epoll model seems to have only one format. Therefore, you only need to refer to the code below to understand epoll, all the code explanations are in the annotations:

While (true)
{
Int NFDs = epoll_wait (m_epoll_fd, m_events, max_events, epoll_time_out); // wait for the occurrence of the epoll event, which is equivalent to listening. As for the relevant port, it must be bound when epoll is initialized.
If (NFDs <= 0)
Continue;
M_bontimechecking = false;
G_curtime = Time (null );
For (INT I = 0; I <NFDs; I ++)
{
Try
{
If (m_events [I]. Data. FD = m_listen_http_fd) // If an HTTP user is detected to connect to the bound http port, create a new connection. Because we use a new socket connection, it is useless.
{
Onaccepthttpepoll ();
}
Else if (m_events [I]. Data. FD = m_listen_sock_fd) // if a new socket user is detected to connect to the bound socket port, create a new connection.
{
Onacceptsockepoll ();
}
Else if (m_events [I]. Events & epollin) // if it is a connected user and receives data, read it.
{
Onreadepoll (I );
}

Onwriteepoll (I); // check whether data needs to be written in the current active connection.
}
Catch (INT)
{
Printf ("Catch catch error/N ");
Continue;
}
}
M_bontimechecking = true;
Ontimer (); // you can perform scheduled operations to delete disconnected users.
}

In fact, according to my current understanding of epoll, that is, the short sections of the Code above, it seems that the times are really different. How can we accept a large number of user connections in the past, but now it's so easy to handle it.

Today, we made a one-day epoll and wanted to create a highly concurrent proxy program. At the beginning, I was so depressed that I couldn't get through it all the time. There were several epoll articles on the Internet. However, I did not go into depth and did not explain some points of attention. As a result, many detours have been taken, so we can share some of our understanding with everyone to avoid detours.

All functions used by epoll are declared in the header file SYS/epoll. H. If you do not understand anything or the function has forgotten it, you can check it out.
The biggest difference between epoll and select is:

When 1epoll is returned, it is clear which sokcet FD has an event and there is no need to compare it one by one. This improves the efficiency.
The fd_setsize of 2select is limited, while epoll is not limited, only related to system resources.

1. epoll_create Function
Function declaration: int epoll_create (INT size)
This function generates a file descriptor dedicated to epoll. In fact, it applies for a space in the kernel to store whether or not the socket FD you want to pay attention to occurs and what happened. Size is the maximum number of socket FD you can pay attention to on this epoll FD. As you have done. As long as you have space. See the difference between select and above. 2.

22. epoll_ctl Function
Function declaration: int epoll_ctl (INT epfd, int op, int FD, struct epoll_event * event)
This function is used to control events on an epoll file descriptor. You can register events, modify events, and delete events.
Parameters:
Epfd: The epoll-specific file descriptor generated by epoll_create;
OP: the operation to be performed, such as registration event. The possible values are epoll_ctl_add registration, epoll_ctl_mod repair and modification, and epoll_ctl_del deletion.

FD: the associated file descriptor;
Event: pointer to epoll_event;
If the call is successful, 0 is returned. If the call is unsuccessful,-1 is returned.

Data Structure Used
Typedef Union epoll_data {
Void * PTR;
Int FD;
_ Uint32_t u32;
_ Uint64_t u64;
} Epoll_data_t;

Struct epoll_event {
_ Uint32_t events;/* epoll events */
Epoll_data_t data;/* User Data variable */
};


For example:
Struct epoll_event EV;
// Set the file descriptor related to the event to be processed
Ev. Data. FD = listenfd;
// Set the event type to be processed
Ev. Events = epollin | epollet;
// Register the epoll event
Epoll_ctl (epfd, epoll_ctl_add, listenfd, & eV );


Common event types:
Epollin: indicates that the corresponding file descriptor can be read;
Epollout: indicates that the corresponding file descriptor can be written;
Epollpri: indicates that the corresponding file descriptor has urgent readable data.
Epollerr: indicates that the corresponding file descriptor is incorrect;
Epollhup: indicates that the corresponding file descriptor is hung up;
Epollet: indicates that an event occurs in the corresponding file descriptor;


3. epoll_wait Function
Function declaration: int epoll_wait (INT epfd, struct epoll_event * events, int maxevents, int timeout)
This function is used to poll the occurrence of I/O events;
Parameters:
Epfd: The epoll-specific file descriptor generated by epoll_create;
Epoll_event: the array used to return the events to be processed;
Maxevents: Number of events that can be processed each time;
Timeout: the timeout value for waiting for an I/O event (I am not sure about the unit);-1 is equivalent to blocking, and 0 is equivalent to non-blocking. Generally, use-1.
Number of events returned.


The usage is as follows:

/* Build the epoll enent for recall */
Struct epoll_event ev_read [20];
Int NFDs = 0; // return the events count
NFDs = epoll_wait (epoll_fd, ev_read, 20,-1 );
For (I = 0; I
{
If (ev_read [I]. Data. FD = sock) // The Listener port Hava data
......

The operating principle of epoll_wait is
The occurrence of the socket FD event registered on the epfd. If so, the sokct FD and event type will be put into the events array.
And clear the socket FD event type registered on epfd. If you want to pay attention to this socket FD in the next loop, you need to use epoll_ctl (epfd, epoll_ctl_mod, listenfd, & eV) to reset the socket FD event type. In this case, epoll_ctl_add is not used, because socket FD is not cleared, but the event type is cleared. This step is very important.
I didn't add this in the beginning, and I spent a morning working on it.

4. A single epoll cannot solve all the problems, especially when each of your operations is time-consuming, because epoll is serialized.
Therefore, it is necessary to establish a thread pool to Improve the efficiency.

//////////////////////////////////////// //////////////////////////////////////
Man provides epoll usage. The example program is as follows:
For (;;){
NFDs = epoll_wait (kdpfd, events, maxevents,-1 );

For (n = 0; n <NFDs; ++ N ){
If (events [N]. Data. FD = listener ){
Client = accept (listener, (struct sockaddr *) & local,
& Addrlen );
If (client <0 ){
Perror ("accept ");
Continue;
}
Setnonblocking (client );
Ev. Events = epollin | epollet;
Ev. Data. FD = client;
If (epoll_ctl (kdpfd, epoll_ctl_add, client, & eV) <0 ){
Fprintf (stderr, "epoll set insertion error: FD = % d/N ",
Client );
Return-1;
}
}
Else
Do_use_fd (events [N]. Data. FD );
}
}
In this case, the et mode is used, that is, edge trigger, which is similar to Level Trigger. Edge trigger in epoll means to only notify new data, if the old data in the kernel buffer is not notified, the following cycle should be used in the do_use_fd function to read the data in the kernel buffer.
While (1 ){
Len = Recv (*******);
If (LEN =-1 ){
If (errno = eagain)
Break;
Perror ("Recv ");
Break;
}
Do something with the recved data ........
}

The above example does not explain how to deal with listen socket FD. Sometimes two threads are used, one for listening to accept and the other for listening to epoll_wait. If so, the listen socket FD uses the default blocking method. If epoll_wait and accept are in the same thread, that is, all are listened by epoll_wait, you need to set the listen socket FD to non-blocking. In this way, you should also use the while packet for accept (similar to the Recv above), because epoll_wait only indicates that there is a connection when returning, it is not said that there are several connections, and epoll_wait in et mode will no longer return because the last connection has not been read. This situation does exist, I spent more than a day on this issue. It should be noted that each call to accept reads a connection from the queue header in the connection queue in the kernel, in a concurrent access environment, multiple connections may arrive at the same time. Oll_wait returns only once.

The only trouble is that epoll can work in two ways: Lt and ET.

LT (Level triggered) is the default working method, and supports both block and no-block socket. in this way, the kernel tells you whether a file descriptor is ready, and then you can perform Io operations on this ready FD. If you do not perform any operation, the kernel will continue to inform you, so the possibility of programming errors in this mode is lower. The traditional select/poll model is representative of this model.

Et (edge-triggered) is a high-speed operating method that only supports no-block socket. In this mode, when the descriptor is never ready, the kernel tells you through epoll. Then it will assume that you know that the file descriptor is ready and will not send more ready notifications for that file descriptor, until you do some operations, the file descriptor is no longer ready (for example, you are sending, receiving, or receiving requests, or an ewouldblock error occurs when the number of data sent and received is less than a certain amount ). However, please note that if I/O operations are not performed on this FD all the time, the kernel will not send more notifications (only once), but in the TCP protocol, more benchmark validation is still required for the acceleration utility of et mode.