Use libevent and libev to improve network application performance

Use libevent and libev to improve network application performance

Use libevent and libev to improve network application performance

Manage multiple UNIX Network Connections

Martin C. Brown, writer, FreelanceMartin Brown has been a professional writer for more than eight years. He is the author of a wide range of books and articles. He specializes in a variety of development languages and platforms-Perl, Python, Java, JavaScript, Basic, Pascal, Modula-2, C, C ++, Rebol, Gawk, Shellscript, Windows, Solaris, Linux, BeOS, Mac OS/X, etc.-also involves Web programming, system management, and integration. Martin is a Microsoft theme expert (SME) and regular contributor to ServerWatch.com, LinuxToday.com, and IBM developerWorks. He is also a formal Blog for Computerworld, The Apple Blog, and other sites. You can contact him through his Web site http://www.mcslp.com.

 

Introduction:Building modern server applications requires some method to receive hundreds, thousands, or even tens of thousands of events at the same time, whether they are internal requests or network connections, they must be processed effectively. There are many solutions, but the libevent library and libev library can greatly improve performance and event processing capabilities. In this article, we will discuss the basic structures and methods used to use and deploy these solutions in UNIX applications. Both libev and libevent can be used in high-performance applications, including applications deployed in the IBM Cloud or Amazon EC2 environment that need to support a large number of concurrent clients or operations.

Release date:February 09, 2011
Level:Intermediate
Original language:English
Access:32236 views
Comment:0 (View | Add comment-Log On)

Average score (39 scores)
Score for this article

 

Introduction

One of the biggest problems faced by many server deployment (especially web Server deployment) is that it must be able to handle a large number of connections. Whether to build cloud-based services to process network communication streams, distribute applications on IBM Amazon EC instances, or provide high-performance components for websites, you must be able to process a large number of concurrent connections.

A good example is that web applications are becoming increasingly dynamic recently, especially those using AJAX technology. If the system to be deployed allows thousands of clients to update information directly on the webpage, such as a system that provides real-time monitoring of events or problems, the speed of providing information is very important. In a grid or cloud environment, persistent connections from thousands of clients may be opened at the same time. Each client must be able to process and respond to requests.

Before discussing how libevent and libev handle multiple network connections, let's briefly review the traditional solutions for handling such connections.

Back to Top

Process multiple clients

There are many different traditional methods to process multiple connections, but they often cause problems when dealing with a large number of connections, because they use too much memory or CPU, or have reached an operating system limit.

The main method is as follows:

• Loop: In the early days, the system used a simple loop selection solution, that is, to traverse the list of open network connections and determine whether data to be read is available. This method is both slow (especially as the number of connections increases) and inefficient (because other connections may be sending requests and waiting for responses when processing the current connection ). When the system cyclically traverses each connection, other connections have to wait. If there are 100 connections, and only one of them has data, it is still necessary to process 99 other connections in order to be able to process the connections that really need to be processed.
• Poll, epoll, and variants: This is an improvement to the loop method. It uses a structure to store the arrays of each connection to be monitored. when data is found on the network socket, the processing function is called through the callback mechanism. The problem with poll is that this structure will be very large. When a new network connection is added to the List, modifying the structure will increase the load and affect the performance.
• Select:select()Function calls use a static structure, which is hardcoded into a relatively small number (1024 connections) in advance, so it is not suitable for very large deployment.

There are other implementations on various platforms (such as/dev/poll on Solaris or kqueue on FreeBSD/NetBSD). They may have better performance on their respective operating systems, but cannot be transplanted, it may not be able to solve the high-level problem of processing requests.

All the above solutions use a simple loop to wait and process requests, and then assign the requests to another function to process the actual network interaction. The key is that loop and network socket require a large amount of Management Code so that different connections and interfaces can be monitored, updated, and controlled.

Another way to process many connections is to use the multi-thread support in the modern kernel to listen and process connections and start a new thread for each connection. This puts the responsibility directly to the operating system, but will increase the overhead in RAM and CPU, because each thread needs its own execution space. In addition, if each thread is busy processing network connections, context switching between threads will be frequent. Finally, many kernels are not suitable for processing such a large number of active threads.

Back to Top

Libevent Method

The libevent library is not replaced.select(),poll()Or other mechanisms. Instead, a package is added to the implementation using the most efficient high-performance solution for each platform.

To process each request, the libevent Library provides an event mechanism, which acts as the package at the underlying network backend. The event system makes it easy to add processing functions for connections and reduces the underlying I/O complexity. This is the core of the libevent system.

Other components of the libevent library provide other functions, including buffered Event Systems (used to buffer data sent to/received from clients) and core implementations of HTTP, DNS, and RPC systems.

The basic method for creating a libevent server is to register the function that should be executed when an operation occurs (such as accepting a connection from the client), and then call the main event Loopevent_dispatch(). The control of the execution process is now handled by the libevent system. After registering an event and calling a function, the event system becomes autonomous. When an application is running, you can add (Register) or delete (cancel registration) Events in the event queue. Event registration is very convenient. You can use it to add new events to process new connections and build a flexible network processing system.

For example, you can open a listening socket and register a callback function.accept()This callback function is called when a new connection is opened, and a network server is created. The code snippet shown in Listing 1 describes the basic process:

Listing 1. Open the listening socket and register a callback function (whenever you need to callaccept()Function to open a new connection) to create a network server.

int main(int argc, char **argv){... ev_init(); /* Setup listening socket */ event_set(&ev_accept, listen_fd, EV_READ|EV_PERSIST, on_accept, NULL); event_add(&ev_accept, NULL); /* Start the event loop. */ event_dispatch();}

event_set()Function to create a new event structure,event_add()Add events to the event queue mechanism. Then,event_dispatch()Start the event queue system and start listening (and accepting) requests.

Listing 2 provides a more complete example to build a very simple echo server:

Listing 2. Building a simple echo server

#include <event.h>#include <sys/types.h>#include <sys/socket.h>#include <netinet/in.h>#include <arpa/inet.h>#include <string.h>#include <stdlib.h>#include <stdio.h>#include <fcntl.h>#include <unistd.h>#define SERVER_PORT 8080int debug = 0;struct client { int fd; struct bufferevent *buf_ev;};int setnonblock(int fd){ int flags; flags = fcntl(fd, F_GETFL); flags |= O_NONBLOCK; fcntl(fd, F_SETFL, flags);}void buf_read_callback(struct bufferevent *incoming, void *arg){ struct evbuffer *evreturn; char *req; req = evbuffer_readline(incoming->input); if (req == NULL) return; evreturn = evbuffer_new(); evbuffer_add_printf(evreturn,"You said %s\n",req); bufferevent_write_buffer(incoming,evreturn); evbuffer_free(evreturn); free(req);}void buf_write_callback(struct bufferevent *bev, void *arg){}void buf_error_callback(struct bufferevent *bev, short what, void *arg){ struct client *client = (struct client *)arg; bufferevent_free(client->buf_ev); close(client->fd); free(client);}void accept_callback(int fd, short ev, void *arg){ int client_fd; struct sockaddr_in client_addr; socklen_t client_len = sizeof(client_addr); struct client *client; client_fd = accept(fd, (struct sockaddr *)&client_addr, &client_len); if (client_fd < 0) { warn("Client: accept() failed"); return; } setnonblock(client_fd); client = calloc(1, sizeof(*client)); if (client == NULL) err(1, "malloc failed"); client->fd = client_fd; client->buf_ev = bufferevent_new(client_fd, buf_read_callback, buf_write_callback, buf_error_callback, client); bufferevent_enable(client->buf_ev, EV_READ);}int main(int argc, char **argv){ int socketlisten; struct sockaddr_in addresslisten; struct event accept_event; int reuse = 1; event_init(); socketlisten = socket(AF_INET, SOCK_STREAM, 0); if (socketlisten < 0) { fprintf(stderr,"Failed to create listen socket"); return 1; } memset(&addresslisten, 0, sizeof(addresslisten)); addresslisten.sin_family = AF_INET; addresslisten.sin_addr.s_addr = INADDR_ANY; addresslisten.sin_port = htons(SERVER_PORT); if (bind(socketlisten, (struct sockaddr *)&addresslisten, sizeof(addresslisten)) < 0) { fprintf(stderr,"Failed to bind"); return 1; } if (listen(socketlisten, 5) < 0) { fprintf(stderr,"Failed to listen to socket"); return 1; } setsockopt(socketlisten, SOL_SOCKET, SO_REUSEADDR, &reuse, sizeof(reuse)); setnonblock(socketlisten); event_set(&accept_event, socketlisten, EV_READ|EV_PERSIST, accept_callback, NULL); event_add(&accept_event, NULL); event_dispatch(); close(socketlisten); return 0;}

The following describes the functions and their operations:

• main(): The main function is created to listen to the connection socket, and then createaccept()To process each connection through the event processing function.
• accept_callback(): When the connection is accepted, the event system calls this function. This function accepts the connection to the client. It adds the client socket information and a bufferevent structure. In the event structure, it adds a callback function for the read/write/error event on the client socket; transmits the client structure (and embedded eventbuffer and client socket) as parameters ). Call the corresponding callback function whenever the corresponding client socket contains read, write, or error operations.
• buf_read_callback(): Called when the client socket has data to be read. As the echo service, this function writes "you said..." back to the client. The socket is still open and can accept new requests.
• buf_write_callback(): Call the data to be written. In this simple service, this function is not required, so the definition is empty.
• buf_error_callback(): Call it when an error occurs. This includes client disconnection. In all scenarios where an error occurs, close the client socket, delete the event entries of the client socket from the event list, and release the memory of the client structure.
• setnonblock(): Set network sockets to enable I/O.

When the client is connected, add new events in the event queue to process client connections. when the client is disconnected, delete the events. Behind the scenes, libevent processes network sockets, identifies the clients that require services, and CALLS corresponding functions respectively.

To build this application, you need to compile the C source code and add the libevent Library:$ gcc -o basic basic.c -levent.

From the client perspective, this server only sends back any text sent to it (see listing 3 ).

Listing 3. The server sends the text sent to it back

$ telnet localhost 8080Trying 127.0.0.1...Connected to localhost.Escape character is '^]'.Hello!You said Hello!

Such network applications are suitable for large-scale distributed deployment that requires processing multiple connections, such as the IBM Cloud system.

It is difficult to observe the handling of a large number of concurrent connections and performance improvement through a simple solution. You can use embedded HTTP to help you understand scalability.

Back to Top

Use the built-in HTTP Server

If you want to build a local application, you can use a general network-based libevent interface. However, an increasingly common scenario is to develop HTTP-based applications, and web pages that load or dynamically reload information. If you use any AJAX library, the client requires HTTP, even if the information you return is XML or JSON.

The HTTP implementation in libevent is not a substitute for the Apache HTTP Server, but a practical solution suitable for large-scale dynamic content associated with the cloud and web environment. For example, you can deploy libevent-based interfaces in IBM Cloud or other solutions. Because HTTP can be used for communication, the server can be integrated with other components.

To use the libevent service, you need to use the same basic structure as the main network event model, but you must also process network interfaces. the HTTP package will handle this for you. This makes the entire process into four function calls (initialize, start the HTTP server, set the HTTP callback function, and enter the event loop), plus the callback function that sends the returned data. Listing 4 provides a very simple example:

Listing 4. Simple example of using libevent Service

#include <sys/types.h>#include <stdio.h>#include <stdlib.h>#include <unistd.h>#include <event.h>#include <evhttp.h>void generic_request_handler(struct evhttp_request *req, void *arg){ struct evbuffer *returnbuffer = evbuffer_new(); evbuffer_add_printf(returnbuffer, "Thanks for the request!"); evhttp_send_reply(req, HTTP_OK, "Client", returnbuffer); evbuffer_free(returnbuffer); return;}int main(int argc, char **argv){ short http_port = 8081; char *http_addr = "192.168.0.22"; struct evhttp *http_server = NULL; event_init(); http_server = evhttp_start(http_addr, http_port); evhttp_set_gencb(http_server, generic_request_handler, NULL); fprintf(stderr, "Server started on port %d\n", http_port); event_dispatch(); return(0);}

The preceding example shows the basic structure of the Code, which does not need to be explained. The main element isevhttp_set_gencb()Function (it sets the callback function to be used when an HTTP request is received) andgeneric_request_handler()The callback function itself (it fills the Response Buffer with a simple message indicating success ).

The HTTP package provides many other functions. For example, a request parser extracts query parameters from a typical request (just like processing CGI requests ). You can also set the handler to be triggered in different request paths. By setting different callback and processing functions, you can use the path '/db/' to provide interfaces to the database, or use the '/memc' to provide interfaces to memcached.

Another feature of the libevent toolkit is the support for universal timers. Events can be triggered after a specified time period. By combining the timer and HTTP, You can provide lightweight services to automatically provide the file content and update the returned data when modifying the file content. For example, in the past, to provide instant update services during frequent news events, front-end web applications need to reload press releases on a regular basis, and now content can be easily provided. The entire application (and web Service) is in the memory, so the response is very fast.

This is the main purpose of the example in listing 5:

Listing 5. Use a timer to provide instant update during frequent news activities

#include <sys/types.h>#include <stdio.h>#include <stdlib.h>#include <string.h>#include <unistd.h>#include <sys/stat.h>#include <event.h>#include <evhttp.h>#define RELOAD_TIMEOUT 5#define DEFAULT_FILE "sample.html"char *filedata;time_t lasttime = 0;char filename[80];int counter = 0;void read_file(){ int size = 0; char *data; struct stat buf; stat(filename,&buf); if (buf.st_mtime > lasttime) { if (counter++) fprintf(stderr,"Reloading file: %s",filename); else fprintf(stderr,"Loading file: %s",filename); FILE *f = fopen(filename, "rb"); if (f == NULL) { fprintf(stderr,"Couldn't open file\n"); exit(1); } fseek(f, 0, SEEK_END); size = ftell(f); fseek(f, 0, SEEK_SET); data = (char *)malloc(size+1); fread(data, sizeof(char), size, f); filedata = (char *)malloc(size+1); strcpy(filedata,data); fclose(f); fprintf(stderr," (%d bytes)\n",size); lasttime = buf.st_mtime; }}void load_file(){ struct event *loadfile_event; struct timeval tv; read_file(); tv.tv_sec = RELOAD_TIMEOUT; tv.tv_usec = 0; loadfile_event = malloc(sizeof(struct event)); evtimer_set(loadfile_event, load_file, loadfile_event); evtimer_add(loadfile_event, &tv);}void generic_request_handler(struct evhttp_request *req, void *arg){ struct evbuffer *evb = evbuffer_new(); evbuffer_add_printf(evb, "%s",filedata); evhttp_send_reply(req, HTTP_OK, "Client", evb); evbuffer_free(evb);}int main(int argc, char *argv[]){ short http_port = 8081; char *http_addr = "192.168.0.22"; struct evhttp *http_server = NULL; if (argc > 1) { strcpy(filename,argv[1]); printf("Using %s\n",filename); } else { strcpy(filename,DEFAULT_FILE); } event_init(); load_file(); http_server = evhttp_start(http_addr, http_port); evhttp_set_gencb(http_server, generic_request_handler, NULL); fprintf(stderr, "Server started on port %d\n", http_port); event_dispatch();}

The basic principle of this server is the same as the previous example. First, the script sets an HTTP server, which only responds to requests that combine the basic URL host/port (do not process the request URI ). The first step is to load the file (read_file()). Use this function when loading the initial file and when the timer triggers the callback.

read_file()Function usagestat()Function call checks the file modification time. It only reads the file content after the last file is modified. This function is called by callingfread()Load file data, copy the data to another structure, and then usestrcpy()Transfers data from the loaded string to the global string.

load_file()A function is a function called when a timer is triggered. It callsread_file()Load the content, and then use the RELOAD_TIMEOUT value to set the timer as the number of seconds before trying to load the file. The libevent timer uses the timeval structure and allows the timer to be specified in seconds and milliseconds. The timer is not periodic. When a timer event is triggered, it is set and then deleted from the event queue.

Compile the code in the same format as the preceding example:$ gcc -o basichttpfile basichttpfile.c -levent.

Now, create a static file used as data. The default file is sample.html, but you can specify any file by using the first parameter on the command line (see Listing 6 ).

Listing 6. Create a static file used as data

$ ./basichttpfileLoading file: sample.html (8046 bytes)Server started on port 8081

Now, the program can accept the request, and the reload timer is started. If sample.html is modified, the file is reloaded and a message is recorded in the log. For example, the output in listing 7 shows the initial load and two reloads:

Listing 7. The output shows the initial load and two reloads.

$ ./basichttpfileLoading file: sample.html (8046 bytes)Server started on port 8081Reloading file: sample.html (8047 bytes)Reloading file: sample.html (8048 bytes)

Note: To maximize the benefits, you must ensure that the environment does not limit the number of opened file descriptors. You can use the ulimit command to modify the limits (requires proper permissions or root access ). The specific settings depend on your OS, but can be used in Linux-nOption to set the number of opened file descriptors (and network sockets:

Listing 8. Use-nOption to set the number of opened file descriptors

$ ulimit -n1024

Raise the limit by specifying a number:$ ulimit -n 20000.

You can use performance benchmarks such as Apache iis2 (ab2) to test the server performance. You can specify the number of concurrent queries and the total number of requests. For example, if you run a benchmark test with 100,000 requests, the number of concurrent requests is 1000:$ ab2 -n 100000 -c 1000 http://192.168.0.22:8081/.

Run the example system using the 8 K file shown in the server example, and the result is about 11,000 requests per second. Keep in mind that this libevent server runs in a single thread, and a single client is unlikely to put pressure on the server because it is also limited by the method for opening requests. Even so, such a processing rate is surprising for single-threaded applications when the document size of the switch is moderate.

Back to Top

Implementation in other languages

Although C language is suitable for many system applications, C language is not often used in modern environments, and the script language is more flexible and practical. Fortunately, most scripting languages such as Perl and PHP are written in C, so C libraries such as libevent can be used through the extension module.

For example, listing 9 shows the basic structure of the Perl network server script.accept_callback()The function is the same as the accept function in the core libevent example shown in Listing 1.

Listing 9. Basic Structure of Perl web server script

my $server = IO::Socket::INET->new( LocalAddr => 'localhost', LocalPort => 8081, Proto => 'tcp', ReuseAddr => SO_REUSEADDR, Listen => 1, Blocking => 0, ) or die $@;my $accept = event_new($server, EV_READ|EV_PERSIST, \&accept_callback);$main->add;event_mainloop();

Libevent implementations written in these languages generally support the core of the libevent system, but do not necessarily support HTTP wrappers. Therefore, it is complicated to use these solutions for scripting applications. There are two methods: either embedding the scripting language into a C-based libevent application, or using one of the many HTTP implementations built based on the scripting language environment. For example, Python contains a highly functional HTTP server class (httplib/httplib2 ).

It should be pointed out that nothing in the scripting language can be re-implemented with C. However, considering the development time constraints, integration with existing code may be more important.

Back to Top

Libev Library

Like libevent, The libev system is also based on the event loop.poll(),select()Based on local implementation of other mechanisms, event-based loops are provided. By the time I wrote this article, libev has lower overhead and can achieve better benchmark test results. Libev APIs are relatively primitive and do not have an HTTP wrapper. However, libev supports more built-in event types. For example, an evstat implementation can monitor attribute changes of multiple files and can be used in the HTTP file solution shown in Listing 4.

However, the basic processes of libevent and libev are the same. Create the required network listening socket, register the event to be called during execution, and then start the main event loop to let the rest of the libev process.

For example, you can use the Ruby interface to provide the echo server in a similar way as listing 1, as shown in listing 10.

Listing 10. Using the Ruby interface to provide the echo server

require 'rubygems'require 'rev'PORT = 8081class EchoServerConnection < Rev::TCPSocket def on_read(data) write 'You said: ' + data endendserver = Rev::TCPServer.new('192.168.0.22', PORT, EchoServerConnection)server.attach(Rev::Loop.default)puts "Listening on localhost:#{PORT}"Rev::Loop.default.run

Ruby is particularly well implemented because it provides wrapper for many common network solutions, including HTTP clients, OpenSSL, and DNS. Other script language implementations include fully functional Perl and Python implementations. You can give it a try.

Back to Top

Conclusion

Both libevent and libev provide flexible and powerful environments and support high-performance network (and other I/O) interfaces for processing server or client requests. The goal is to support thousands or even tens of thousands of connections in an efficient (low CPU/RAM usage) manner. In this article, you have seen some examples, including the built-in HTTP service in libevent, which can be used to support web applications based on IBM Cloud, EC2, or AJAX.