Linux epoll introduction and program example

1. Where is epoll sacred?

Epoll is currently a hot candidate for developing large-scale concurrent network programs in Linux. epoll is officially introduced in the linux2.6 kernel, similar to select, in fact bothI/O Multiplexing TechnologyAnd there is nothing mysterious.

In fact, there is always no way to design concurrent network programs in Linux, such as the typical Apache model (process per connection (PPC) and TPC (thread perconnection) model, as well as the select model and poll model, why should we introduce epoll? That's something to talk about...

2. disadvantages of common models

If the disadvantages of other models are not shown, how can we compare the advantages of epoll.

2.1 PPC/TPC Model

These two models share the same idea:Let every incoming connection work on its own. Don't bother me anymore.. Only PPC opens a process for it, while TPC opens a thread. But don't bother me. There is a price. It requires time and space. When there are too many connections, so many processes/threads will switch, and this overhead will come up; therefore, the maximum number of connections that this type of model can accept is not high, generally about several hundred.

2.2 select model

1. maximum number of concurrent threads, because the FD opened by a process (file descriptor) is limited, which is set by fd_setsize. The default value is 1024/2048, therefore, the maximum concurrency of the select model is limited accordingly. Modify fd_setsize by yourself? Although the idea is good, let's take a look at the following...

2. efficiency problems: each select call will linearly scan all FD sets, which will result in a linear decline in efficiency. The consequence of increasing the fd_setsize is that everyone is coming slowly. What? All timeout ??!!

3. How does the kernel notify the user of the FD message when copying the kernel/user space memory? On this issue, select adopts the memory copy method.

2.3 poll Model

Basically, the efficiency is the same as that of select. The two and three disadvantages of select are not modified.

3. epoll Improvement

I criticized other models one by one. Let's take a look at epoll's improvements. In fact, the disadvantages of select are the advantages of epoll.

3.1. epoll does not have the maximum number of concurrent connections. The maximum number is the maximum number of files that can be opened. This number is generally greater than 2048,Generally, this number has a great relationship with the system memory.The number can be viewed in CAT/proc/sys/fs/file-max.

3.2. The biggest advantage of epoll is its efficiency improvementOnly your "active" ConnectionsAnd has nothing to do with the total number of connections. Therefore, in the actual network environment, epoll is much more efficient than select and poll.

3.3. memory copy, epoll uses"Shared Memory", This memory copy is also omitted.

 

4. Why is epoll efficient?

The efficiency of epoll is inseparable from the design of its data structure, which will be mentioned below.

First, let's recall the select model. When an I/O event arrives, select notifies the application that an event has been processed quickly, and the application must Round-Robin all FD sets, test whether an event occurs in each FD and process the event. The Code is as follows:

Int res = select (maxfd + 1, & readfds, null, null, and 120 );

If (RES> 0)

{

For (INT I = 0; I <max_connection; I ++)

{

If (fd_isset (allconnection [I], & readfds ))

{

Handleevent (allconnection [I]);

}

}

}

// If (RES = 0) handle timeout, Res <0 handle error

 

Epoll not only tells the application that there is an I/0 event, but also tells the application-related information, which is filled by the application, therefore, based on this information, the application can directly locate the event without traversing the entire FD set.

Int res = epoll_wait (epfd, events, 20,120 );

For (INT I = 0; I <res; I ++)

{

Handleevent (events [N]);

}

5. epoll Key Data Structure

As mentioned above, epoll is fast and closely related to its data structure. Its key data structure is:

Structepoll_event {

_ Uint32_t events; // epoll events

Epoll_data_t data; // user datavariable

};

Typedef Union epoll_data {

Void * PTR;

Int FD;

_ Uint32_t u32;

_ Uint64_t u64;

} Epoll_data_t;

It can be seen that epoll_data is a Union struct, which can store many types of information such as FD and pointer. With it, the application can directly locate the target.

6. Use epoll

Since epoll is better than select, how can it be used? Will it be complicated... Let's take a look at the three functions below to see how easy epoll is to use.

Int epoll_create (INT size );

Generating an epoll-specific file descriptor is actually applying for a kernel space to store whether or not the socket FD you want to pay attention to occurs and what events have occurred. Size is the maximum number of socket FD you can pay attention to on this epoll FD. The size is customized as long as the memory is sufficient.

Int epoll_ctl (INT epfd, intop, int FD, structepoll_event
* Event );

Controls events on an epoll file descriptor: Registration, modification, and deletion. The epfd parameter is the file descriptor used to create epoll for epoll_create. Compared with the fd_set and fd_clr macros in the select model.

Int epoll_wait (INT epfd, structepoll_event * events, int maxevents, int timeout );

Wait for the occurrence of an I/O event. parameter description:

Epfd:Epoll_create ()The generated epoll-specific file descriptor;

Epoll_event: the array used to return the events to be processed;

Maxevents: Number of events that can be processed each time;

Timeout: the timeout value for waiting for an I/O event;

Number of events returned.

Compared with the select function in the select model.

7. Example Program

The following is an example of a simple echo server program. Although it is small and dirty, it also contains a simple timeout check mechanism. For the sake of simplicity, no error handling is performed.

/// A simple echo server using epoll in Linux // 2013-03-22: several problems have been modified. 1 is the/n format, 2. Remove the et mode accidentally added to the original code. // It is a simple program, it is determined that the buffer offset when Recv/send is added // by sparkling // # include <sys/socket. h> # include <sys/epoll. h> # include <netinet/in. h> # include <ARPA/inet. h> # include <fcntl. h> # include <unistd. h> # include <stdio. h> # include <errno. h> # include <iostream> using namespace STD; # define max_ev Ents 500 struct myevent_s {int FD; void (* call_back) (int fd, int events, void * Arg); int events; void * ARG; int status; // 1: in epoll wait list, 0 not in char buff [128]; // Recv data buffer int Len, s_offset; long last_active; // last active time }; // set event void eventset (myevent_s * eV, int FD, void (* call_back) (INT, Int, void *), void * Arg) {ev-> FD = FD; ev-> call_back = call_back; ev-> event S = 0; ev-> Arg = ARG; ev-> Status = 0; bzero (ev-> buff, sizeof (ev-> buff )); ev-> s_offset = 0; ev-> Len = 0; ev-> last_active = Time (null);} // Add/MoD an event to epoll void eventadd (INT epollfd, int events, myevent_s * eV) {struct epoll_event EPV = {0, {0}; int op; EPV. data. PTR = EV; EPV. events = ev-> events = events; If (ev-> Status = 1) {op = epoll_ctl_mod;} else {op = epoll_ctl_add; ev-> Status = 1 ;} If (epoll_ctl (epollfd, op, ev-> FD, & EPV) <0) printf ("event add failed [FD = % d], evnets [% d] \ n ", ev-> FD, events); else printf (" event add OK [FD = % d], OP = % d, evnets [% 0x] \ n ", ev-> FD, op, events);} // delete an event from epoll void eventdel (INT epollfd, myevent_s * eV) {struct epoll_event EPV = {0, {0 }}; if (ev-> status! = 1) return; EPV. data. PTR = EV; ev-> Status = 0; epoll_ctl (epollfd, epoll_ctl_del, ev-> FD, & EPV);} int g_epollfd; myevent_s g_events [max_events + 1]; // g_events [max_events] is used by listen FD void recvdata (int fd, int events, void * Arg); void senddata (int fd, int events, void * Arg ); // accept new connections from clients void acceptconn (int fd, int events, void * Arg) {struct sockaddr_in sin; sock Len_t Len = sizeof (struct sockaddr_in); int NFD, I; // accept if (NFD = accept (FD, (struct sockaddr *) & sin, & Len )) ==- 1) {If (errno! = Eagain & errno! = Eintr) {} printf ("% s: Accept, % d", _ FUNC __, errno); Return ;}do {for (I = 0; I <max_events; I ++) {If (g_events [I]. status = 0) {break;} if (I = max_events) {printf ("% s: Max connection limit [% d]. ", _ FUNC __, max_events); break;} // set nonblocking int iret = 0; If (iret = fcntl (NFD, f_setfl, o_nonblock) <0) {printf ("% s: fcntl nonblocking failed: % d", _ FUNC __, iret); break;} // Add a r EAD event for receive data eventset (& g_events [I], NFD, recvdata, & g_events [I]); eventadd (g_epollfd, epollin, & g_events [I]);} while (0); printf ("New conn [% s: % d] [Time: % d], POS [% d] \ n", inet_ntoa (sin. sin_addr), ntohs (sin. sin_port), g_events [I]. last_active, I);} // receive data void recvdata (INT FD, int events, void * Arg) {struct myevent_s * EV = (struct myevent_s *) ARG; int Len; // receive data Len = r ECV (FD, ev-> buff + ev-> Len, sizeof (ev-> buff)-1-ev-> Len, 0); eventdel (g_epollfd, Ev ); if (LEN> 0) {ev-> Len + = Len; ev-> buff [Len] = '\ 0'; printf ("C [% d]: % s \ n ", FD, ev-> buff); // change to send event eventset (EV, FD, senddata, Ev); eventadd (g_epollfd, epollout, ev);} else if (LEN = 0) {close (ev-> FD); printf ("[FD = % d] POS [% d], closed gracefully. \ n ", FD, ev-g_Events);} else {close (ev-> FD); printf (" rec V [FD = % d] error [% d]: % s \ n ", FD, errno, strerror (errno ));}} // send data void senddata (int fd, int events, void * Arg) {struct myevent_s * EV = (struct myevent_s *) ARG; int Len; // send data Len = Send (FD, ev-> buff + ev-> s_offset, ev-> len-ev-> s_offset, 0); If (LEN> 0) {printf ("send [FD = % d], [% d <-> % d] % s \ n", FD, Len, ev-> Len, ev-> buff); ev-> s_offset + = Len; If (ev-> s_offset = ev-> Len) {// change Receive event eventdel (g_epollfd, Ev); eventset (EV, FD, recvdata, Ev); eventadd (g_epollfd, epollin, Ev );}} else {close (ev-> FD); eventdel (g_epollfd, Ev); printf ("send [FD = % d] error [% d] \ n", FD, errno) ;}} void initlistensocket (INT epollfd, short port) {int listenfd = socket (af_inet, sock_stream, 0); fcntl (listenfd, f_setfl, o_nonblock ); // set non-blocking printf ("server listen FD = % d \ n", listenf D); eventset (& g_events [max_events], listenfd, acceptconn, & g_events [max_events]); // Add listen socket eventadd (epollfd, epollin, & g_events [max_events]); // bind & listen sockaddr_in sin; bzero (& sin, sizeof (SIN); sin. sin_family = af_inet; sin. sin_addr.s_addr = inaddr_any; sin. sin_port = htons (port); BIND (listenfd, (const sockaddr *) & sin, sizeof (SIN); Listen (listenfd, 5);} int main (INT argc, char ** Argv) {unsigned short Port = 12345; // default port if (argc = 2) {Port = atoi (argv [1]);} // create epoll g_epollfd = epoll_create (max_events); If (g_epollfd <= 0) printf ("create epoll failed. % d \ n ", g_epollfd); // create & bind listen socket, and add to epoll, set non-blocking initlistensocket (g_epollfd, Port ); // event loop struct epoll_event events [max_events]; printf ("server running: Port [% d] \ N ", Port); int checkpos = 0; while (1) {// a simple timeout check here, every time 100, better to use a mini-heap, and add timer event long now = Time (null); For (INT I = 0; I <100; I ++, checkpos ++) // doesn't check listen FD {If (checkpos = max_events) checkpos = 0; // recycle if (g_events [checkpos]. status! = 1) continue; long duration = now-g_events [checkpos]. last_active; If (duration> = 60) // 60 s timeout {close (g_events [checkpos]. FD); printf ("[FD = % d] timeout [% d -- % d]. \ n ", g_events [checkpos]. FD, g_events [checkpos]. last_active, now); eventdel (g_epollfd, & g_events [checkpos]) ;}// wait for events to happen int FDS = epoll_wait (g_epollfd, events, max_events, 1000 ); if (FDS <0) {printf ("epoll_wait error, exit \ n"); break;} For (INT I = 0; I <FDS; I ++) {myevent_s * EV = (struct myevent_s *) events [I]. data. PTR; If (events [I]. events & epollin) & (ev-> events & epollin) // read event {ev-> call_back (ev-> FD, events [I]. events, ev-> Arg);} If (events [I]. events & epollout) & (ev-> events & epollout) // write event {ev-> call_back (ev-> FD, events [I]. events, ev-> Arg) ;}}// free resource return 0 ;}