Analysis and exploration of Twemproxymemcache protocol--Analysis of Twemproxy code complement

Memcache is a cache server similar to Redis, but Memcache only provides a simple way to store keys and values, and memcache is relatively easy to store in a variety of ways than Redis supports. Memcache through TCP or UDP connections to achieve memcache client and server interaction. The Memcache protocol is self-defined and divided into two types: a text protocol (which is the focus of our discussion today) and a binary protocol (in the scope of our discussion today) that only describes the Memcache protocol for communicating with TCP connections and text protocols. If there is any improper description, please point out.

Memcache Text Protocol

Store command

The add/replace/set/append/prepend command format is as follows:

<command name> <key> <flags> <exptime> <bytes>\r\n

<data block>\r\n

The format is described below:

<command name> can be add (that is, add a non-existent key-value pair), replace (that is, replace the existing key-value pair), set (both of the above features), append (add the corresponding content after the value of the existing key), and prepend ( Add the corresponding content before the value corresponding to the key that exists.

<key> is the key name of the data item.

<flags> is to save any 16-bit unsigned shaping (written in decimal) on the server along with the data and send blocks, typically 0, when retrieving content.

<exptime> is a valid time. If 0, the entry never expires, and if not 0, the item is deleted after <exptime>.

<bytes> is the length of the <data block>, which is the length of the data item.

<data block> is data for data items.

add/replace/set/append/prepend Command reply

"Stored\r\n" indicates that the storage was successful.

"Not_stored\r\n" indicates that the data was not stored, but not because an error occurred. This usually means add, the condition of replace is not satisfied, or the item is already in the delete queue (refer to the "delete" command later in this article).

The CAS command format is as follows:

CAS <key> <flags> <exptime> <bytes> <cas unique> [noreply]\r\n

<data block>\r\n

<key>, <flags>, <exptime>, <bytes> and <data block> with add/replace/set/append/prepend command

<cas Unique> is a globally unique 64-bit number associated with an existing data entry. The client should use the value returned by the "gets" command to perform a "CAS" update operation.

[noreply] is optional to indicate do not reply. Note: If the request line is malformed, the server may not be able to reliably parse the "noreply" option. In this case, it may send an error message to the client, which can cause problems if the client does not read the information. The client should only construct legitimate requests.

The CAS command replies as follows:

"exist\r\n" indicates that the data to be updated has been modified since your last fetch.

"No_found\r\n" indicates that the data to be modified does not exist.

The delete Format command is as follows:

Delete <key> <time>\r\n

Delete Deletes the corresponding key name.

<key> is the key that needs to be deleted name.

<time> is a unit of time in seconds during which the server rejects the "add" and "replace" commands for this key name. At this point the content is placed in the delete queue and can no longer be obtained by "get", nor with the "add" and "replace" commands (but the "set" command is available). Until the specified time, the content is eventually purged from the server's memory.

The delete command replies as follows:

"deleted\r\n" indicates successful execution

"not_found\r\n" indicates that the content was not found

Read command

The get/gets command format is as follows:

Get/gets <key>*\r\n

<key>* refers to multiple key names that are separated by spaces.

The Get/gets command responds as follows:

VALUE <key> <flags> <bytes> [<cas unique>]\r\n

<data block> \ r \ n

<key> is the key name of the data item

<flags> is the flags set by the store command, typically 0

<bytes> is the length of the data block

<cas Unique> is a 64-bit integer that uniquely identifies a particular data item.

<data block> is data for data items.

All the data here ends with "end".

The incr/desc Command format is as follows:

Incr/desc <key> <value> [noreply]\r\n

<key> is the key name of the data item

<value> is a value that incr (increments)/desc (decrements) the data item. It is a 64-bit unsigned decimal integer.

[Norply] is an optional parameter, do not reply.

incr/desc Command reply:

"Not_found\r\n" indicates that the data item cannot be found.

"<value>\r\n", where <value> is the new value of this data item after the increment/decrement operation.

Parsing of requests for memcache text protocols

This is the Memcache_parse_req function in proto/memcache.c.

It is also a finite state machine to complete the analysis

1     enum {2 Sw_start,3 Sw_req_type,4 Sw_spaces_before_key,5 Sw_key,6 Sw_spaces_before_keys,7 Sw_spaces_before_flags,8 Sw_flags,9 Sw_spaces_before_expiry,Ten Sw_expiry, One Sw_spaces_before_vlen, A Sw_vlen, - Sw_spaces_before_cas, - Sw_cas, the Sw_runto_val, - Sw_val, - Sw_spaces_before_num, - Sw_num, + Sw_runto_crlf, - Sw_crlf, + sw_noreply, A sw_after_noreply, at Sw_almost_done, - Sw_sentinel -} state;

the finite state machine diagram here is as follows:

Storage is the storage command, that is, the Add/replace/set/append/prepend/cas command, Retreval is the read command, that is, the get\gets command, other than the storage command, Get\gets command, Touch command, INCR/DESC commands other than the command.

For example, the Get\gets command, first Sw_req_type, is the get\gets string, then the Sw_spaces_before_key, that is, the space, then Sw_key, is the key name, and then back to Sw_spaces_before_key, Constantly reciprocating.

Analysis of the reply of memcache text protocol

This is the MEMCACHE_PARSE_RSP function in proto/memcache.c.

It is also a finite state machine to complete the resolution, the following are these states.

1    enum {2 Sw_start,3 Sw_rsp_num,4 Sw_rsp_str,5 Sw_spaces_before_key,6 Sw_key,7Sw_spaces_before_flags,/*5*/8 Sw_flags,9 Sw_spaces_before_vlen,Ten Sw_vlen, One Sw_runto_val, ASw_val,/*Ten*/ - Sw_val_lf, - Sw_end, the Sw_runto_crlf, - Sw_crlf, -Sw_almost_done,/* the*/ - Sw_sentinel +} state; 

The finite state machine diagram here is as follows:

For example, if the reply to get and gets goes first to Sw_rsp_str, "VALUE", then goes to Sw_spaces_befoer_key line, knows the end of the line Sw_runto_crlf, and then enters the data block sw_runto_ Val this line, know Sw_rsp_str, until sw_rsp_str to parse out "END", will go to sw_crlf this line.

Summarize

This article mainly introduces the text protocol of memcache and how tweproxy resolves memcache request packet and memcache reply package.

Analysis and exploration of Twemproxymemcache protocol--Analysis of Twemproxy code complement