Assembly-based implementation of A/C + + co-process (for servers)

This article is an implementation of the C/s + + co-process. We need to achieve these two goals:

1. Sequential thinking with synchronized server programming for functional design and code debugging-I used the LIBCO part of the process

2. Performance with asynchronous I/O-I use the event I/O in libevent Apache PHP mysql

Structure, is to combine the functions of both LIBCO and libevent, so I named my project libcoevent, meaning "libevent-based synchronization Process Server programming framework." The meaning of CO in the name does not mean LIBCO, but coroutine.

In the programming language, I chose C + +, mainly because LIBCO only supports Linux based on x86 or x64 architecture, and this architecture is basically a PC, or a resource-free, performance-friendly embedded system, and C + + has no problem at all. This article explains the principles of code implementation.

If you want to use the project, include three options in the link Options-lco -levent -lcoevent.

Class relationships and basic functions

Class relationships

Class inheritance Relationships

The basic inheritance diagram for a class is as follows:

In the actual invocation, only the classes on the leaf nodes of the inheritance tree are actually used, and the other classes are treated as virtual classes.

Class dependent relationships

All kinds of instances are subordinate in the program running, except as the Base class of the top layer, other leaf classes must be attached to the other classes in the running environment to execute. The dependency graph is as follows:

• The base class provides the most basic operating environment and manages the Server objects;

• The Procedure object manages the Client object. The diagram shows that both the Server and Session objects Manage Client objects.

• The Server object is created and initialized by the application to run in the Base object. You can configure the automatic destruction of server objects when the server ends or when its dependent Base objects are destroyed.

• The Session object is created automatically by the Server object in session mode, and is invoked by the application's specified program portal, when the session ends (function callreturn) or its subordinate when the server object service ends, it is automatically destroyed by the server object.

• The Client object is created by an interface that the application calls the Procedure object to interact with the third-party service. The application can call the interface to destroy the Client object in advance, or it can be destroyed automatically at the end of the Procedure service.

Base and Event classes

The Base class is used to run the individual services of libcoevent. Each instance of the base class should correspond to a thread, and all services run in the base instance in a co-process manner. From the known, thebase class contains an object of the Libevent libraryevent_baseand a series of Event objects of the libraries.

The event class actually borrows the name of the Libeventstruct event, because each instance of the event class corresponds to an object of Libeventevent. The focus we need to focus on is Procedure and the Client class.

Procedure class

There are two key features of the Procedure class:

1. Each object has a LIBCO, which has its own independent contextual information that can be used to write a separate server process (procedure);

2. Subclasses of Procesure can create Client objects to communicate and interact with third-party servers.

The Procedure class has two subclasses, Server and Session, respectively.

Server class

The Server class is created and initialized by the application to run in the Base object. There are three subclasses of the Server class:

• subroutine: does not actually act as any server program, but provides the most basicsleep()functions and supports the ability of the Procedure class to create Client objects, so the application can be used as a temporary create or resident internal program.

• udpserver: After the application creates and initializes the Udpserver object, the program is automatically bound to a datagram socket interface. Applications can implement network services by sending and receiving packets in a network interface. Udpserver provides both Normal mode and session mode .

• tcpserver: After the application creates and initializes the Tcppserver object, the program automatically binds and listens to the stream socket. TCPServer only supports session mode .

The so-called " Normal mode ", which is the entry function of the application to register the server object, and the behavior of the server object is manipulated by the application.

The so-called " session mode ", refers to the Udpserver or TCPServer object, after receiving the incoming data, automatically differentiate the client, and separate creation Session object for processing. Each Session object serves only one client.

Session class

The session object cannot be actively created by the app, but is automatically created on demand by the Server class in session mode. The Session object is characterized only by communication with a single client (as opposed to a Udpserver object), so there is nosend()function, onlyreply().

Thecoevent.hsession class and its subclasses declared in the header file are pure virtual classes in order to prevent the application from explicitly building the session object and hiding the implementation details.

Client class

The Client object is created by the Procedure object and reclaimed by the Procedure object. The purpose of the Client object is to proactively initiate communication to the remote server. Since this action belongs to the client from the perspective of the customer-service structure, it is named clients.

Dnsclient

In particular, the Dnsclient class is the subclass of the Client, and the class exists to solve the blocking problem in asynchronous I/Ogetaddrinfo(). The implementation of Dnsclient is described in the code and my previous article, "DNS message structure and personal DNS resolution code implementation."

For the Dnsclient class, the implementation of the principle is to encapsulate a UdpClient object, through the object to complete the sending and receiving of the DNS message, and in the class to achieve the resolution of the message.

udpserver--implementation of Libevent based on the co-process

The principle of udpserver generic mode is a very typical libevent-based synchronous coprocessor Server framework. Its code implementation, the core function is the following several functions:

• _libco_routine(), the entry function of the association, using this function, transforms into a uniform service entry function for Liboevent

• _libevent_callback(),libevent time callback function, in this function, to achieve the recovery of the context of the process.

• UDPServer::recv_in_timeval(), the data receive function, in this function, realizes the key data wait function, simultaneously realizes the association context the preservation

The total number of code for the above three functions, plus the empty line is not more than 200 lines, I believe it is easy to see. The following are specific explanations of the implementation principle:

Libco Coprocessor Interface

As I said earlier, I am using LIBCO as a co-libraries. The process is transparent to the application, but for the implementation of the library, this is the core.

The following explains some of the interfaces provided by LIBCO's co-function (the number of LIBCO documents is simply "touching", which is also often used on the internet ... ）：

Create and destroy a co-process

Libco uses a structstruct stCoRoutine_t *to save the process, and callsco_create()to create a co-process object, using theco_release()destroy Coprocessor resource.

Enter the co-process

After the co-process has been created, the callco_resume()can begin the execution of the process from the beginning of the co-function.

Pause the co-process

When the co-process is required to hand over the CPU usage, it can invoke theco_yield()release process and switch off the context. After the call, the context reverts to the last callco_resume()'s thread.co_yield()the location of the call can be treated as a " breakpoint ."

Recovery co-process

The function used to recover the co-process and to create the process isco_resume()to call the function and switch the current stack to the context of the specified thread, which resumes execution from the " breakpoint " mentioned above.

Implementation of co-scheduling

As you can see from the previous section, the LIBCO function function we used, although it contains the switch function of the process, but when to switch, after switching the CPU allocation, this is what we need to implement and encapsulate the work.

The time to create and destroy a co-process is naturally when the Udpserver class initializes and reconstructs. The following focuses on the operation of the Enter, pause, and resume processes:

Enter the co-process

The code to enter/resume the co-process is in_libevent_callback(), and there is a line:

// handle control to user application
co_resume(arg->coroutine);

If the current process has not been executed, after executing this code, the program switches to the execution of the co-function specified when the LIBCO process was created. For Udpserver, which is the_libco_routine()function. This function is very simple and only has three rows:

static void *_libco_routine(void *libco_arg)
{
    struct _EventArg *arg = (struct _EventArg *)libco_arg;
    (arg->worker_func)(arg->fd, arg->event, arg->user_arg);
    return NULL;
}

Converts the LIBCO callback function to the server function execution specified by the application by passing in parameters.

But how do you implement the first Libevent callback? This is still simple, just set the time-out to 0 when calling Libevent,event_add()which causes the Libevent event to time out immediately. With this mechanism, we implement the Procedure service functions immediately after the Base is run.

Pause and resume co-processes

When the call is the focus of theco_yieldimplementation of this process, the location of the callco_yieldis a place that may lead to context switching, as well as the key technical point of translating the asynchronous programming framework into a synchronization framework. You can refer to the udpserver recv_in_timeval()function here. The basic logic of the function is as follows:

One of the most important branches is the judgment of the Libevent event flag, and the most important logic is the invocation of theevent_add()co_yield()function. The function fragment is as follows:

struct timeval timeout_copy;
timeout_copy.tv_sec = timeout.tv_sec;
timeout_copy.tv_usec = timeout.tv_usec;
    ...
event_add(_event, &timeout_copy);
co_yield(arg->coroutine);

Here, weco_yield()interpret the function as a breakpoint, and when the program executes here, the CPU's right to use is surrendered and the program returns toco_resume()the upper-level function of the call. Where exactly is this "upper-level function"? This is actually the function mentioned earlier_libevent_callback().

From_libevent_callback()the point of view, the programco_resume()returns from the function and continues to execute. At this point we can understand that: the scheduling of the association, is actually borrowedlibeventto carry out. Here we are going to lookco_resume()at a few words above:

// switch into the coroutine
if (arg->libevent_what_ptr) {
    *(arg->libevent_what_ptr) = (uint32_t)what;
}

Here, the libevent event flag value is passed to the co-process, which is an important basis for the previous event judgment. When the time arrives, the_libevent_callback()co_resume()CPU use right is handed back to the coprocessor at the location called below.

Destroy the co-process

In additionci_yield()to this, the process function callreturnwill also cause theco_resume()return from, so in the_libevent_callback()we also need to determine whether the process has ended. If the process is completed, the associated resources should be destroyed. Seeif (is_coroutine_end(arg->coroutine)) {...}code in the condition body.

Session mode

In the implementation of this project, a server design pattern called "session mode" is provided. The session mode refers to the Udpserver or TCPServer object, which automatically differentiates the client after receiving the incoming data and creates a session object to be processed separately. Each Session object serves only one client.

For TCPServer , the implementation of the above function is relatively simple, because after listening to a TCP socket, when there is an incoming connection, as long as the callaccept(), you can get a new file descriptor, for this file descriptor to create a new the subclass of the Server is OK-this is the tcpsession class.

But udpserver is more troublesome, because UDP can't do this. We can only implement the so-called session on our own.

Udpsession implementation

Design goals

We need to implement the following effect of the Udpsession class:

• When a class calls the recv function, it receives only the data that is sent from the corresponding remote client

• Class calls the send function (the actual implementation isreply()), you can use the udpserver port to reply

Recv ()

In engineering,Udpsession is an abstract class, and the actual implementation is udpitnlsession. But to be precise,udpitnlsession 's realization is closely dependent on udpserver. In this section, you can refer to the_session_mode_worker()do-while()Loop Body code in the Udpserver function. Program ideas are as follows:

• Udpserver maintains a udpsession dictionary with a combination of remote IP + port names as key.

• When the data arrives, determine if the combination of the remote IP + port is in the dictionary, and if so, the data is copied to the corresponding session, and if it does not exist, the session is created.

Copy the code for the data, see the function implementation of the Udpitnlsession classforward_incoming_data().

Reply ()

Send the data is actually very simple, directly to the Udpserver FD to proceedsendto().

Quit

For the server object in session mode, the code provides a function that can be called by its session, which requires the server to exit and destroy the resource:quit_session_mode_server(). The implementation principle is to trigger an event to the serverEV_SIGNAL. For normal I/O events, this should not occur, and we live here as exit signals. If the server discovers this signal, it triggers the exit logic.

Application examples

The sample code for this project is divided into two parts: server and client, where Server uses libcoevent, and client is simply a simple program written in Python . This article does not describe the client part of the code.

The code for the server, respectively, for the server class of three sub-classes to do a sample application. Using logic that includes blank lines, debug statements, error judgments, and less than 300 rows, a process and two services are implemented. It should be said that logic is clear, and it saves a lot of code.

Subroutine

Through the function_simple_test_routine(), a one-time linear network logic is presented. Program, routine first creates a dnsclient object, requests a domain name to the default name server, and thenconnect()the server's port 80. After success, return directly.

This function shows the usage scenarios of subroutine and the use of Client objects, especially the easy-to-use method of dnsclient .

Udpserver

The entry function of Udpserver is_udp_session_routine()that the function is to provide the client with a domain name lookup service. Clients sends a string as the domain name to be queried, and then the server returns the query results to the client after the dnsclient object request.

This function shows the (more complex and complete) use of udpsession objects and dnsclient .

TCPServer

The entry function is_tcp_session_routine(), the logic is relatively simple, mainly shows the usage of tcpsession .

Postscript

In principle, thelibcoevent has been developed, has realized the necessary function, can be used to write the server program completely. Of course because this is the first edition, so a lot of code seems a bit messy. The significance of this library is that, from the teaching point of view, it is possible to explain the principle of the realization of the more origin of the C + + process, and it can also be used as a usable co-server library.

Readers are welcome to criticize the library and welcome new demands from readers-for example, I decided to add a few needs, which is TODO:

1. Implement Httpserver, as tcpserver Subclass, provide HTTP fcgi service;

2. Implement the Sslclient class to handle external SSL requests.

Related articles:

C # Network Programming series (eight) UdpClient implementation of synchronous UDP server

C Language Implementation PHP server

Related videos:

C # Tutorials