RBL Development NOTE 3

20:06:24

Today, we are developing this epoll to handle network events and encapsulate a more robust epoll model to handle basic network Io.

1) The topic timed out is not obtained first.

When epoll is developed to include select/poll type Io replay, there are several programming techniques: [The following are all for TCP protocol. If UDP is required in future development, this programming technology will be extended]

1Types of file descriptors to process

A) listenfd = socket (); this listenfd is the listenfd of the listener. Now we use it as a total Io file. In fact, this listenfd is an array index. The listenfd readble status monitored by epoll is required.TcpaccepthandlerBecause listening to the corresponding core stuff is a series of raw client information:

Among them, ip fd port is used as a fully functional program ip fd port, which is used as raw information and a series of RBL server-side control information. This heavyweight data structure of rblclient is constructed, in this way, the network layer and the RBL service layer can be combined through this data structure. Tcpaccepthandler is currently doing this.

B) The rblclient is constructed in stage A, but this operation only performs the raw accpet operation of TCP. However, real logical operations need to be introduced, that is, all the data communication received by rblserver, whether on its own or on the client. The following code uses typedef void fileproc (epolleventloop * EP, void * data, int FD) to register a universal function pointer for each descriptor of the file to be processed. This memory consumption is required. This does not take into account the registration event. In fact, here we need to handle readable and writeable events. 1) The Readable event is delivered to other services through this layer. 2) The Readable event can be written to another component [itself] and handed over to the network terminal.

2. Create an event processing model that can be inserted at any time.

In implementation, We can insert events at any time like redis. For example, after creating an epollloop object, We can insert a listenfd to the epollloop object after N steps.

After processing a client command, I need to tell him how to do it through a callback event. At the same time, the del function is required because the client is suspended at any time or automatically leaves. These seem obvious but need to be carefully encapsulated. [Injection in redisReadqueryfromclient/SendreplytoclientThese two event functions: I still use this mode in the face of such an excellent redis design, but where can I dive into these two functions? You need to know it carefully.

 

What protocol does it define? I use the same protocol mode for redis compatibility.

A simple class is defined as follows:

 

Class epolleventloop {public: epolleventloop (INT maxclient _): maxclient (maxclient _), maxepollevent (maxclient _), maxfd (-1) {epollfd = epoll_create (2048 ); if (epollfd =-1) STD: cout <"epoll create failed! "<STD: Endl;} void creatfileevent (int fd, fileevent * prere_event); // register the event void deletefileevent (int fd, int mask ); // here is the deletion event void processevents (INT flag); // flag indicates the type of event. Here is the read/write event ~ Epolleventloop () {} PRIVATE: int maxfd; // The maximum number of files currently registered eventloop is initialized to-1std: vector <fileevent> fileevent; // STD :: vector stores int epollfd; // epollfd file descriptor STD: vector <struct epoll_event> maxepollevent; int maxclient ;};

 

Batch processing is a cumbersome implementation of the redis protocol:

　　

* <Parameter quantity> cr lf $ <parameter 1 byte quantity> cr lf <parameter 1 DATA> Cr lf... $ <number of bytes of parameter n> cr lf <data of parameter n> CR LF
In this case, the three functions (Readqueryfromclient/Sendreplytoclient/Tcpaccepthandler) Is based on rblclient as the main parameter, decoding or encryption are inReadqueryfromclient/SendreplytoclientFunction completed.
There is another difficulty in implementation: The data buffer introduced here is not enough when the code is added or decoded. What should I do? It cannot be put into structured data all at once, so we should make a tag bit during implementation. It is really troublesome to read it multiple times. In this case, we need to add several more structures in rblclient to control the amount. Crying Network Protocol Part Of The http://www.redisdoc.com/en/latest/topic/protocol.html to refer to this to simplify it after all, there is no redis command so much

The following is a headache: The above implementation version is the single-thread mode:
How can we implement multiple threads?
Epoll thread: In the workqueue of the task queue, what is added to this workqueue? In fact, each rblclient is copied to the workqueue, and the consumption thread calls the takequeue ()
Formalization:

LOCK1()TakeQUEUE()UNLOCK1()…………LOCK2()CommandExecute()UNLOCK2()

Implementation of lock1Lock-free queue

For lock2, there is a commandexecute () Here, it controls a part of the bit array. So for me, if we useSpin_lock ()Will locking be better? This need to be evaluated

 

In addition, for consistency, this operation mode may not immediately get the desired result for a single customer. This is required because a time window is not strongly consistent. I sacrifice strong consistency here, but if workqueue is always very consistent if it is small, this time window will certainly be very small and will not have a great impact on the application.

The implementation part of tomorrow is estimated to be the most difficult thread control part, which is planned to be completed in two to three days. The part of eventloop code implementation and the part of protocol implementation will be posted first, and then the ideas will be discussed in a series.
First, bye.

  

 

RBL Development NOTE 3