Notes on establishing memcached Client Connection

When memcached initializes server_sockets during the startup process, the system determines whether to perform TCP or UDP listening Based on the startup parameters. Here, only the TCP is concerned.

During initialization, server_socket will apply to the system for configuring the address, bind, listen, and other operations after listening to the socket. The next key step is
A Conn is created for the listening socket, which is used to describe the context of a client request. Obviously, memcached treats the listening socket as a conn to facilitate management,
Main Code:

/* Set the initial state of conn to conn_listening, indicating that this is a listening socket. In the subsequent event state machine, it is used to receive user connections */If (! (Listen_conn_add = conn_new (SFD, conn_listening, ev_read | ev_persist, 1, transport, main_base) {fprintf (stderr, "failed to create listening connection \ n "); exit (exit_failure);} listen_conn_add-> next = listen_conn; listen_conn = listen_conn_add;/* listen_conn is a global variable used to save all listener connections */

Comments of some fields in the conn data structure:

Struct conn {int SFD;/** connect to the corresponding FD, that is, listen to fd or the user connects FD */char * rbuf; /** store the READ command */char * rcurr;/** location of the resolved rbuf */INT rsize; /** full rbuf length */INT rbytes;/** number of unresolved data records starting from rcurr */char * wbuf; char * wcurr; int wsize; int wbytes; /** identifies the next state after the current State ends. In the state machine, use */Enum conn_states write_and_go; void * write_and_free; /** free this memory after finishing writing */char * ritem;/** stores the value in key-value, copy the received data to the storage location in the state machine */INT rlbytes; /* data structure used to read value * // * data for the nread state * // *** item is used to hold an item structure created after reading the command * line of Set /Add/replace commands, but before we finished reading the actual *. the data is read into item_data (item) to avoid extra copying. * // *** when the value data in the key-value is not read, item is used to store the Item1 data structure generated by processing the set/Add/replace command locks, * after reading the value, it will be directly read into the data area of Item1 to prevent multiple data copies.
*/Void * item;/* these commands set/Add/replace will use * // * data for the swallow state */INT sbytes; /* How many bytes to swallow */Enum protocol;/* Identify the transmitted data type char or binary */Enum network_transport transport; /* identify the data transmission method TCP or UDP or UNIX socket */INT hdrsize;/* Number of headers 'worth of space is allocated */bool noreply; /* indicates whether the Command needs to reply */Conn * Next;/* points to the next connection to form a single-chain table structure */libevent_thread * thread;/* points to the thread, each user connection is allocated to a worker thread */};

The key code in new_conn identifies the initial state of the conn and the processing function:

C-> state = init_state;/** set the initial state of the connection */... event_set (& C-> event, SFD, event_flags, event_handler, (void *) C);/** set event listening and event_handler callback function */event_base_set (base, & C-> event);/** register to libevent */

Event_handler performs simple FD verification and transfers the event to drive_machine for processing. This is the implementation of the memcached event processing state machine,
Let's take a look at some logic of drive_machine connection:

Static void drive_machine (conn * c) {bool stop = false; int SFD; socklen_t addrlen; struct sockaddr_storage ADDR; int nreqs = settings. reqs_per_event; int res; const char * STR; Assert (C! = NULL); While (! Stop) {Switch (c-> state) {Case conn_listening:/** the listener connection whose status is conn_listening is responsible for receiving the client connection, receive the connection and set it to non-blocking */accept (c-> SFD, (struct sockaddr *) & ADDR, & addrlen); fcntl (SFD, f_setfl, fcntl (SFD, f_getfl) | o_nonblock) <0)/** determine whether the maximum number of connections is set */If (settings. maxconns_fast & stats. curr_conns + stats. reserved_fds> = settings. maxconns-1 ){...} if else {/** does not reach the maximum number of connections, allocate the connection and create a conn to indicate that the initial state of the new Conn is conn_ne. * // ** W_cmd: listens for read events. The transmission protocol is tcp * // **. The general logic of skipping to dispach_conn_new -------- */dispatch_conn_new (SFD, conn_new_cmd, ev_read | ev_persist, data_buffer_size, tcp_transport);} Stop = true;/** jump out of the while loop */break;/** the status below is basically driven by the event after the client establishes a connection */case... :... return;}/*** this function can only be called by the main thread and is used to allocate the newly received connection to the worker thread */void dispatch_conn_new (int sfd, Enum conn_states init_state, int event_flags, int read_buffer_size, Enum Network_transport transport) {cq_item * Item = cqi_new ();/* cq_item is the data structure of the data interaction between the master thread and the worker thread. It encapsulates a connection */Char Buf [1]; if (item = NULL) {close (SFD);/* given that malloc failed this may also fail, but let's try */fprintf (stderr, "failed to allocate memory for connection object \ n"); return;} int tid = (last_thread + 1) % settings. num_threads;/** Robin-round loop to obtain a target worker thread */libevent_thread * thread = threa DS + tid ;... cq_push (thread-> new_conn_queue, item);/** press the new connection packaging entity to the woker thread to process the connection queue */memcached_conn_dispatch (SFD, thread-> thread_id ); buf [0] = 'C '; /** the main thread sends a message 'C' to the worker through the pipe channel created during the preliminary test thread, indicating that a new connection is allocated */If (write (thread-> policy_send_fd, buf, 1 )! = 1 ){...}} /*** after the worker thread receives a message, it calls back the callback function * thread_libevent_process of the event that pipe follows when the thread starts. Next let's continue to look at the function logic */static void thread_libevent_process (int fd, short which, void * Arg) {libevent_thread * Me = ARG; cq_item * item; char Buf [1];/** read message */If (read (FD, Buf, 1 )! = 1) if (settings. verbose> 0) fprintf (stderr, "can't read from libevent PIPE \ n");/** determine the Message Type */switch (BUF [0]) {Case 'C':/** new connection */item = cq_pop (Me-> new_conn_queue ); /** pop the New Connection Structure pushed by the main thread */If (null! = Item) {/** create a new connection me-> base indicates the libevent event listening structure that registers the event of interest to the connection */Conn * c = conn_new (item-> SFD, item-> init_state, item-> event_flags, item-> read_buffer_size, item-> transport, me-> base );... cqi_free (item);} break; /** the following two message identifiers are mainly related to thread startup registration * // * We were told to flip the lock type and report in */case 'l ': me-> item_lock_type = item_lock_granular; register_thread_initialized (); break; Case 'G': Me-> item_lock_type = item_lock_global; register_thread_initialized (); break ;}}

The new connection is registered with the callback function event_handler when conn_new is created. event_handler transfers event processing to the drive_machine state machine,

That is, the logic of other states of drive_machine is driven by the command event sent by the client. The above is the process of establishing the client connection, drive_machine state machine.
Comments for other statuses are reserved for the next note.