Memcached Source Analysis--Command flow analysis

I. Execution of orders

The first is to start the memcached with the following parameters:

<span style= "FONT-SIZE:18PX;" >-p <num>      Set TCP port number (default: 11211)-u <num>      UDP Listening port (default: 11211, 0 o'clock off)-L <ip_addr>  bound address ( Default: All allow, regardless of internal or external network or local replacement IP, there is a security risk, if set to 127.0.0.1 can only be native access)-C <num>      Max simultaneous connections (default:1024)-D            Run as Daemon-u <username> bind using specified to run process <username>-m <num>      allow maximum memory usage, Unit m (default: + MB)-P < File>     writes the PID to the file <file>, which allows for fast process termination behind, and needs to be used with-D </span>

#$:./usr/local/bin/memcached-d-U root-l 192.168.10.156-m 2048-p 12121

The client connects over the network:

Telnet 192.168.10.156 12121

Then you can manipulate the commands, and the common commands are as follows:

<span style= "FONT-SIZE:18PX;" >setaddreplacegetdelete</span>

The format is as follows:

<span style= "FONT-SIZE:18PX;" >command <key> <flags> <expiration time> <bytes><value> parameters are described below: command set/add/ Replacekey     key to find cached values flags     can include integer parameters for key-value pairs, which the client uses to store additional information about key-value pairs expiration     time (in seconds, 0) for saving key-value pairs in the cache Represents forever) bytes The     value stored in a byte point stored in the cache     (always on the second row) </span>

Second, the command execution Process Code analysis

First look at the command data structure in the worker thread:

/**
* The structure representing a connection into memcached.
*/
typedef struct Conn Conn;


A few parameters are very important:
char * rbuf: Used to store commands in client data messages.
The size of the int rsize:rbuf.
char * Rcurr: The character pointer of the unresolved command.
int rbytes: The length of the command to parse.


The structure is as follows:

<span style= "FONT-SIZE:18PX;"   >struct conn {int sfd;   Char *rbuf;  /** buffer to read commands into */char *rcurr;   /** But if we parsed some already, the is where we stopped */int rsize;  /** Total Allocated size of RBUF */int rbytes;      /** how much data, starting from Rcur, does we have unparsed */* data for the mwrite state */struct IOVEC *iov;   int iovsize;   /* Number of elements allocated in iov[] */int iovused;      /* Number of elements used in iov[] */struct MSGHDR *msglist;   int msgsize;   /* Number of elements allocated in msglist[] */int msgused;   /* Number of elements used in msglist[] */int msgcurr;  /* element in msglist[] being transmitted now */int msgbytes; /* Number of bytes in current msg */Libevent_thread *thread; /* Pointer to the thread object serving this connection */...}; </span>

state Machine Migration: Drive_machine (conn *c)

At a fairly level, citing the author's http://calixwu.com/, he is no longer drawn.


Describe the overall state machine process in text:
1. When the client and memcached establish a TCP connection, memcached listens to the client for a new connection and has readable data based on the event events of Libevent.
2. When the client has command data message escalation, it will trigger the case state of Conn_read in the Drive_machine method.
3. Memcached reads the client's message through the Try_read_network method. If the read fails, it returns conn_closing, closes the client's connection, and if no data is read, it returns conn_waiting, waits for the client's event to arrive, and exits the Drive_machine loop, if the data is read successfully. The status is forwarded to Conn_parse_cmd processing, and the read data is stored in the C->rbuf container.
4. Conn_parse_cmd's main job is to parse commands. This method is mainly used to read the command data in C->rbuf, and to separate the command of the data message by \ n Try_read_command. If the data in the C->BUF memory block does not match \ n, then the command data message that continues to wait for the client arrives conn_waiting, otherwise it is forwarded to the Process_command method to handle the specific command (the command parsing is delimited by the-\ Symbol).
5. Process_command is mainly used to deal with specific commands. Tokenize_command This method is very important, the command is disassembled into multiple elements (the maximum length of key 250). For example, we take the get command as an example, we will eventually jump to the Process_get_command command Process_*_command this series is to deal with the specific command logic.
6. We enter the Process_get_command, and when the data processing is completed, it will be forwarded to the Conn_mwrite state. If the fetch data fails, the connection is closed.
7. After entering Conn_mwrite, the main way is to submit data to the client via the transmit method. If the write data fails, the connection is closed or the Drive_machine loop is exited, and if the write succeeds, it is transferred to the Conn_new_cmd state.
8. Conn_new_cmd This state is primarily to handle the remaining commands in the c->rbuf. The main look at Reset_cmd_handler this method, this method goes back to determine whether there are remaining messages in the c->rbytes is not processed, if not processed, then transferred to Conn_parse_cmd (fourth Step) continue to parse the remaining commands; To Conn_waiting, awaiting the arrival of new events. The Conn_shrink method is executed once each time before the transfer.
9. Conn_shrink method is mainly used to deal with the command message container c->rbuf and output content of the container is the data full? Whether the size of the buffer needs to be enlarged and whether the block of memory needs to be moved. Accepts command message initialization memory block size 2048, maximum 8192.

Three, a brief analysis of the following code
1, read-write event callback function: Event_handler, this method is finally called Drive_machine

void Event_handler (const int FD, const short which, void *arg) {conn* c = (conn *) Arg;drive_machine (c);}

Drive_machine:
Drive_machine in this method, all the logic that needs to be handled is judged by c->state.
Conn_listening: Listening status
Conn_waiting: Wait Status
Conn_read: Read status
Conn_parse_cmd: command-line parsing
Conn_mwrite: Writing data to clients
Conn_new_cmd: Parsing a new command
//

static void Drive_machine (conn *c) {bool stop = False;while (!stop) {switch (c->state) {case conn_waiting://through update_e The vent function confirms whether it is a read state, and if so, cuts to Conn_readif (!update_event (c, Ev_read | Ev_persist)) {conn_set_state (c, conn_closing);}            Conn_set_state (c, Conn_read);            Stop = true;                       Break           Case Conn_read://Read data and cut to a different state according to read, normally cut to conn_parse_cmd res = try_read_network (c);                Switch (res) {case read_no_data_received:conn_set_state (c, conn_waiting);            Break                Case Read_data_received:conn_set_state (c, Conn_parse_cmd);            Break                Case Read_error:conn_set_state (c, conn_closing);            Break                Case READ_MEMORY_ERROR:/* Failed to allocate more MEMORY */* state already set by Try_read_network */            Break                         } break; Case Conn_parse_cmd:            Read the command and parse the command if the data is not enough then cut to conn_waiting if (Try_read_command (c) = = 0) {/* We need more dat A!          */Conn_set_state (c, conn_waiting);                        } break;            Case conn_mwrite:res = transmit (c); Switch (res) {case Transmit_complete:if (c->state = = Conn_mwrite) {/* xxx:i don ' t k Now, this wasn ' t, the general case */if (C->protocol = = Binary_prot) {conn                    _set_state (c, C->write_and_go);                    } else {//command reply completed, and switch to Conn_new_cmd to process the remaining command parameters Conn_set_state (c, Conn_new_cmd);                        }}} break; ...}}}

The above logic mainly reflects the state machine conversion process, the following focus on the data processing this piece:
Command format: Set username zhuli\r\n get username \ n
The command in the data message is delimited by the \ n line break. Because data packets have sticky and unpacking properties, they wait until the command line is complete
To be parsed. All only matches the \ n symbol to match a complete command.

static int Try_read_command (conn *c) {if (C->protocol = = Binary_prot) {//Binary mode Dispatch_bin_command (c);}          else{//the command to find if there are \n,memcache commands by \ n to split el = MEMCHR (C->rcurr, ' \ n ', c->rbytes);//If you find \ n, there is a complete command in C->rcurr. Cont = el + 1; The pointer node at the beginning of the next command//This side of the judgment is \ r \ n, if it is \ r \ n, then El moves forward an if ((El-c->rcurr) > 1 && * (el-1) = = ' \ r '          ) {el--;                  }///Then the last character of the command is separated by the (string ending symbol) *el = ' + ';                Processing commands, C->rcurr is the command Process_command (c, C->rcurr); Move the pointer node to the next command c->rbytes-= (Cont-c->rcurr); c->rcurr = cont;}} Handle the specific commands. After the command is decomposed, distribute the static void Process_command (conn *c, char *command) {token_t tokens[max_tokens];//split command to a different specific operation: Put the split command element into an array of tokens Ntokens = tokenize_command (command, tokens, max_tokens);//The first parameter of the exploded command is action Method 1, Process_get_ Command (c, tokens, Ntokens, false); "Get"/"bget" 2, Process_update_command (c, tokens, Ntokens, comm, false); "Add"/"Set"/... 3, ProcesS_get_command (c, tokens, ntokens, true); "Gets" ...>> 4-n}

Here with get command under the daytime:

static inline void Process_get_command (conn *c, token_t *tokens ...) {it = Item_get (key, Nkey, c);//Memory store fast block fetch data if (IT) {//Acquired data */* * Construct the response. Each hits adds three elements to the * outgoing data list: * "VALUE" * key * "" + F Lags + "+ data length +" \ r \ n "+ data (with \ r \ n) *///build Initialize the data structure returned Add_iov (c," VALUE ", 6); Add_iov (c, Item_ke Y (it), It->nkey); Add_iov (c, Item_suffix (IT), it->nsuffix-2); Add_iov (c, suffix, suffix_len); Add_iov (c, "end\r\n ", 5);//Last cut to conn_mwrite called transmit function conn_set_state (c, Conn_mwrite);}} /* * Returns an item if it hasn ' t been marked as expired, * lazy-expiring as needed.    */item *item_get (const char *key, const size_t NKEY, conn *c) {Item *it;    uint32_t HV;    HV = Hash (key, nkey);    Item_lock (HV);    it = Do_item_get (key, Nkey, HV, C);    Item_unlock (HV); return it;} Write data to the client. When the data is written, if the write fails, the connection is closed, and if the write succeeds, the state is modified to conn_new_cmd,//continue parsing the remaining commands in c->rbuf static enum transMit_result transmit (conn *c) {//MSGHDR send data structure struct MSGHDR *m = &c->msglist[c->msgcurr]; Sendmsg Send data method res = sendmsg (C->SFD, M, 0); ...}

For processing of the remaining commands:

Reset command handler static void Reset_cmd_handler (conn *c) {c->cmd =-1;      C->substate = bin_no_state;          if (c->item! = NULL) {item_remove (C->item);      C->item = NULL; } conn_shrink (c); This method is to check the size of the C->RBUF container//If the remaining unresolved commands > 0, continue to jump to Conn_parse_cmd parse command if (C->rbytes > 0) {conn_      Set_state (c, Conn_parse_cmd);      } else {//If the command is resolved, continue waiting for new data to arrive Conn_set_state (c, conn_waiting);  }}/* * Shrinks a connection ' s buffers if they ' re too big. This prevents * periodic large "get" requests from permanently chewing lots of server * memory. * * This should is called in between requests since it can wipe output * buffers! */static void Conn_shrink (conn *c) {//Check rbuf size if (c->rsize > Read_buffer_highwat && c->rbytes &l T        data_buffer_size) {char *newbuf;   if (C->rcurr! = c->rbuf) memmove (C->rbuf, C->rcurr, (size_t) c->rbytes);     NEWBUF = (char *) realloc ((void *) c->rbuf, data_buffer_size);            if (newbuf) {c->rbuf = Newbuf;        C->rsize = data_buffer_size;    } C->rcurr = c->rbuf; }...}

for asynchronous socket compilation is the callback + state machine, be sure to write down all the states. There are a few points to pay special attention to:
1. Registered event handlers cannot block or actively sleep, otherwise the entire worker thread is in a suspended state.
2, single thread, but its inherent complexity--the burden of decomposing linear thinking into a bunch of callbacks (breaking up linear thought into a bucketload of callbacks)--still exists
3, the processing of each event needs to maintain a state, the context is closely related, the code should always be careful when writing.
4, pay attention to the working mode of epoll: LT or et mode, usually callback when try to handle more packets.

Memcached Source Analysis--Command flow analysis