Distributed Cache System memcached (ix)--state machine socket connection and distribution

The previous section has analyzed that the main thread listens for socket registration events and the callback function in the worker thread that connects the socket registration event is event_handler, and the core part of Event_handler is a finite state machine: Drive_machine. Therefore, the specific business processing of this state machine will be analyzed in depth.

The memcached encapsulates each socket as a conn structure that contains the file descriptor of the socket, SFD, the Register events event, the connection state structure conn_states, and many other information fields, including the state structure: conn_ The various states of the socket are included in the States. The state machine Drive_machine is the state structure to determine the current state of the socket, so that the business logic processing.

The structure of the connection state is as follows:

<span style= "FONT-SIZE:18PX;"    >//socket the possible state of the struct enum conn_states {conn_listening,//Listening status/**< the socket which listens for connections */  Conn_new_cmd,//prepare for the next connection/**< Prepare connection for next command */conn_waiting,//waiting to read a packet/**< waiting For a readable socket */conn_read,//Read network data/**< reading in a command line */conn_parse_cmd,//parse buffer data/* *< try to parse a command from the input buffer */conn_write,///Easy reply data/**< writing out a simple response * /Conn_nread,//read fixed bytes of network data/**< reading in a fixed number of bytes */conn_swallow,//processing data not required for write buffers/**&lt ;    Swallowing unnecessary bytes w/o storing */conn_closing,//close connection/**< Closing this connection */Conn_mwrite, Write multiple item data sequentially/**< writing out many items sequentially */conn_closed,//connection closed/**< connection is closed * /conn_max_state//MAX state, assert using/**< max (used for assertion) */};</span>

Next look at the general picture of Drive_machine, which is primarily a while loop to handle the business logic of each state:

The core part of the listener socket and connection socket event callback function://Finite state machine: performs the corresponding operation according to the state of the socket conn_sattes static void Drive_machine (conn *c) {bool stop = false;    int sfd;    Socklen_t Addrlen;    struct Sockaddr_storage addr;    int nreqs = settings.reqs_per_event;    int res;    const char *STR;        ASSERT (c! = NULL);//Because there is a conversion or transition between States, it is necessary to loop until the stop is determined and then (!stop) {///for the various states of the socket, the corresponding business process switch (c->state) {            Case conn_listening://Monitor Status Addrlen = sizeof (addr),//////////The main thread enters the state machine after the accept operation, this operation is also non-blocking. SFD = Accept (C-&GT;SFD, (struct sockaddr *) &addr, &addrlen), #endif//connection failure if (SFD = =-1) {/// /}//connection is successful, set the connection socket to non-blocking if (!USE_ACCEPT4) {if (Fcntl (SFD, F_SETFL, F Cntl (SFD, F_GETFL) |                    O_nonblock) < 0) {perror ("setting o_nonblock");                    Close (SFD);                Break }//if the maximum number of connections is exceeded (as determined by the record of the global State structure), you need to close the connection if (Settings.maxconns_fast &Amp;& Stats.curr_conns + Stats.reserved_fds >= settings.maxconns-1) {//// } else {//If there is no overload, direct distribution (UDP, no need to establish a connection, direct distribution) worker thread Dispatch_conn_new (SFD, conn_new_cmd, Ev_read |            Ev_persist, Data_buffer_size, Tcp_transport);            } stop = true;        Break Case Conn_waiting:case Conn_read:case conn_parse_cmd:case conn_nread://and  Other various status return;        }}

This section focuses on the first state conn_listening ：

The state is the main thread that listens to the socket for business processing: listens for sockets, accepts, and sends the resulting connection socket to a selected worker thread.

 Switch (c->state) {case conn_listening://listening status Addrlen = sizeof (addr); #ifdef HAVE_ACCEPT4            if (use_accept4) {sfd = Accept4 (C-&GT;SFD, (struct sockaddr *) &addr, &addrlen, Sock_nonblock);            } else {sfd = accept (C-&GT;SFD, (struct sockaddr *) &addr, &addrlen);            The #else///main thread enters the state machine and performs the accept operation, which is also non-blocking. SFD = Accept (C-&GT;SFD, (struct sockaddr *) &addr, &addrlen), #endif if (sfd = =-1) {if (                    Use_accept4 && errno = = Enosys) {use_accept4 = 0;                Continue } perror (Use_accept4?                "Accept4 ()": "Accept ()");                    if (errno = = Eagain | | errno = = ewouldblock) {/* These is transient, so don ' t log anything */                Stop = true;                     } else if (errno = = emfile) {//connection overload if (Settings.verbose > 0)   fprintf (stderr, "Too many open connections\n");                    Accept_new_conns (FALSE);                Stop = true;                    } else {perror ("accept ()");                Stop = true;            } break; }//connection is successful, set the connection socket to non-blocking if (!USE_ACCEPT4) {if (Fcntl (SFD, F_SETFL, Fcntl (SFD, F_GETFL) |                    O_nonblock) < 0) {perror ("setting o_nonblock");                    Close (SFD);                Break                }//If you exceed the number of simultaneous online maximum connections set (default is 1024) (judging by the record of the global State structure), you need to close the connection if (Settings.maxconns_fast && Stats.curr_conns + Stats.reserved_fds >= settings.maxconns-1) {str = "ERROR Too many open conn                Ections\r\n ";                res = write (sfd, str, strlen (str));                Close (SFD);                Stats_lock ();                stats.rejected_conns++;            Stats_unlock (); } else {//If there is no overload, direct distribution (UDP, no need to establish connection, direct distribution) worker thread Dispatch_conn_new (SFD, conn_new_cmd, Ev_read |            Ev_persist, Data_buffer_size, Tcp_transport);            } stop = true;         Break }

Where the worker threads are selected with polling (Round-robinWay The distribution function that connects the socket is dispath_conn_new:

The main thread is in the callback function of the listener socket, and when a new connection arrives, the new connection to which the function will be accepted is distributed to the worker thread//NOTE: Because UDP does not need to establish a connection, it is distributed directly to the worker thread void Dispatch_conn_new ( int sfd, enum conn_states init_state, int event_flags, int read_buffer_size, enum Network_transport    Transport) {Cq_item *item = cqi_new ();//Get an idle ITEM char Cq_item] from the buf[1 resource pool;        if (item = = NULL) {close (SFD); /* Given that malloc failed this could also fail, but let's try */fprintf (stderr, "failed to allocate memory for con        Nection object\n ");    return; } int tid = (last_thread + 1)% settings.num_threads;//Select a thread by Round-robin algorithm Libevent_thread *thread = threads + tid ;//cache this selected thread last_thread = tid;//update the last selected thread number//set cq_item fields ITEM->SFD = SFD;//SFD is the connection socket Item->init_sta    Te = init_state;    Item->event_flags = Event_flags;    Item->read_buffer_size = read_buffer_size;    Item->transport = transport;//The main thread will post item to the selected worker thread in the item connection queue Cq_push (Thread->new_conn_queue, item); Memcached_conN_dispatch (SFD, thread->thread_id); Buf[0] = ' C '; Pipe notification: The NOTIFY_SEND_FD of the worker thread writes the character C, which indicates that there is a connection if (write (THREAD->NOTIFY_SEND_FD, buf, 1)! = 1) {perror ("writ    ing to thread notify pipe "); }}

As you can see, in this distribution function first extracts an item from the Cq_item resource pool (the idle list), sets the information for each field of the connection socket, then selects a worker thread by polling, and then puts the item in the Connection task queue CQ for that worker.    Finally, the notification message is written by notifying the write end of the pipeline. Next is the work thread that has been analyzed earlier to handle all the business of connecting the socket.

Distributed Cache System memcached (ix)--state machine socket connection and distribution