Memcached Protocol resolution

This article was reproduced from: http://www.ccvita.com/306.html

Agreement
The memcached client communicates with the server using a TCP link . (UDP interface is also valid, refer to the following "UDP protocol") a running Memcached server to monitor some (can be set) port. The client connects these ports, sends commands to the server, reads the response, and finally closes the connection.

You do not need to send any commands to end a session. When the memcached service is no longer required, the client can close the connection at any time. It is important to note that clients are encouraged to cache these connections instead of reopening the connection every time they need to access the data. This is because memcached is deliberately designed to open up a lot of connections in a timely manner and work efficiently (hundreds of, thousands if needed). Caching these connections eliminates the overhead associated with establishing a connection (/*/, in contrast, the overhead of preparing a new connection on the server side can be negligible.) ）。

There are two types of data sent in the Memcache protocol: text lines and free data. The lines of text are used to respond to commands and servers from the client. Free data is used when the client accesses data from the server side. The same server will pass a byte stream back to the free number data. /*/servers do not care about the byte order of free data. There are no restrictions on the characteristics of free data, but by the lines of text mentioned earlier, the recipient (server or client) of this data is able to accurately know the length of the database being sent .

the text line is fixed with "\ r \ n" (the carriage return immediately follows a newline character) . The free data also ends with "\ r \ n", but the \ r (carriage return), \ n (newline character), and any other 8-bit characters, can appear in the data. Therefore, when a client fetches data from the server, it must use the length of the data chunk to determine the end of the data chunk, rather than "\ r \ n" At the end of the data block, even if they are fixed here.

Key value
the data stored in the memcached is identified by a key value . The key value is a text string that must be unique to the client that needs access to the data. The current length limit of the key value is set to 250 characters (of course, the client usually does not use such a long key); tabs and other whitespace characters (such as spaces, line breaks, and so on) cannot be used in key values.

Command
All commands are divided into 3 categories:
the Store command (with 3 items: ' Set ', ' Add ', ' Repalce ') instructs the server to store some data identified by the key value . The client sends a one-line command followed by the data chunk, and then the client waits for a command line from the receiving server to indicate success or not.
The retrieve command (only one item: ' Get ') instructs the server to return data that matches the given key value (one or more key values in a request). The client sends a line of commands, including all the requested key values, and the server sends back to the client a line of information about the content, followed by the corresponding data block, until the server ends with a line of "end" response commands.

/*?*/Other commands are not allowed to carry free data. In these commands, the client sends a single line of commands, and then waits for a row (as determined by the command) to respond, or a multiline command that ends with a line of "end".

A row of commands is fixed to start with the command name, followed by a space-separated argument (if there are parameters). The command name is case sensitive and must be lowercase. Some of the commands that the client sends to the server contain time-frames (actions for content or client requests). At this point, the specific content of the time limit can be either the UNIX timestamp (the number of seconds since January 1, 1970), or the number of seconds the current time begins. For the latter, it cannot exceed 60*60*24*30 (30 days); if exceeded, the server will be interpreted as a Unix timestamp rather than a second offset from the current time.

Error string
Each command sent by the client may receive an error string reply from the server. These error strings appear in three ways:

-"error\r\n"
means that the client sent a non-existent command name.

-"Client_error <error>\r\n"
means that there are some client errors in the input command line, such as entering a non-compliant protocol. <error> is part of a human error-prone interpretation ...

-"Server_error <error>\r\n"
means that some server errors caused the command to fail to execute. The <error> part is a false commentary that is easy for humans to understand. In some serious cases (which should not normally be encountered), the server will close the connection after sending this line of error. This is the only case where the server actively shuts down the connection.
In the description of each subsequent command, these error lines are no longer mentioned in particular, but the client must take into account the possibilities of their existence.

Store command
First, the client sends a line like this:
<command name> <key> <flags> <exptime> <bytes>\r\n
-<command name> is set, add, or Repalce

• Set means "store this data"
• Add means "store this data only when the server does not retain data for this key value "
• Replace means "to store this data only when the server has retained the data for this key value "

-<key> is the key value of the data that is required to be stored by the next client
-<flags> saves any 16-bit unsigned shaping (written in decimal) on the server along with data and send blocks when retrieving content. The client can use it as a "bit field" to store some specific information , which is opaque to the server.
-<exptime> is the end time. If 0, the item never expires (although it may be deleted to make room for other cached items). If it is not 0 (the Unix timestamp or the current time of the second offset), the client cannot obtain the content until the end time is reached.
-<bytes> is the byte length of the subsequent chunk of data, excluding the "\ r \ n" used for the distinction. It can be 0 (followed by an empty data block).

After this line, the client sends a chunk of data.
<data block>\r\n
-<data block> is a large segment of the 8-bit data whose length is specified by the <bytes> in the preceding command line.

After sending the command line and data block, the client waits for a reply, and the possible reply is as follows:
-"stored\r\n"
Indicate success.
-"not_stored\r\n"
Indicates that the data was not stored, but not because an error occurred. This usually means that the condition of the Add or Replace command is not true, or that the item is already in the delete queue (refer to the "delete" command later).

Retrieve command
The row fetch command is as follows:
Get <key>*\r\n
-<key>* represents one or more key values, a string separated by a space
After this line of command, the client waits for 0 or more items, each item receives a line of text, and then follows the data chunk. After all the items have been delivered, the server sends the following strings:
"End\r\n"

To indicate that the response is complete.
The server sends each content in the following form:
VALUE <key> <flags> <bytes>\r\n
<data block>\r\n
-<key> is the name of the key being sent
-<flags> is a token set by the store command
-<bytes> is the length of the subsequent data block, * does not include * its delimiter "\ r \ n"
-<data block> is the data sent

If some key names are sent in the retrieval request, and the server does not return the list of items, this means that the server does not have these key names (either because they have never been stored, or have been deleted, or expire, or have been deleted by the client) to make room for other content.

Delete
The command "Delete" allows content to be removed externally:
Delete <key> <time>\r\n

-<key> is the key name of the content that the client wants the server to delete
-<time> is a unit of time in seconds (or on behalf of Unix time until a moment), during which time the server rejects the "add" and "replace" commands for this key name. At this point the content is placed in the delete queue and can no longer be obtained by "get", nor with the "add" and "replace" commands (but the "set" command is available). Until the specified time, the content is eventually purged from the server's memory.
The <time> parameter is optional and defaults to 0 (indicates that the content is cleared immediately and subsequent storage commands are available).
This command has a line response:
-"deleted\r\n"
Indicates successful execution
-"not_found\r\n"
Indicates that the content was not found

Refer to the subsequent "flush_all" command to invalidate all content

Increase/decrease
The commands "incr" and "DECR" are used to modify the data when some content needs to be increased or decreased. This data must be a decimal 32-bit unsigned integer new. If it is not, it will be treated as a%. The modified content must exist, and when the "incr"/"DECR" command modifies the nonexistent content, it is not treated as 0, but the operation fails.

The client sends the command line:
INCR <key> <value>\r\n
Or
DECR <key> <value>\r\n
-<key> is the name of the content that the client wants to modify
-<value> is the total number of clients to increase/decrease.

Reply to the following centralized scenario:
-"not_found\r\n"
Indicates the value of the item content and does not exist.
-<value>\r\n,<value> is increased/decreased.
Note the "DECR" command is underflow: If the client tries to reduce the result less than 0 o'clock, the result will be 0. The "incr" command does not overflow.

State
The command "stats" is used to query the server's running state and other internal data. There are two types of formats. With no parameters:
stats\r\n
This will then output the status, setting values, and documentation. Another format with some parameters:
Stats <args>\r\n
Through <ARGS>, the server returns various internal data. Because of possible changes at any time, this article does not provide the types of parameters and their return data.

Various states
After being subjected to the "stats" command without parameters, the server sends multiple lines of content, as follows:
STAT <name> <value>\r\n
The server uses the following line to terminate the list:
end\r\n
In each row state,<name> is the name of the state,<value> the data that makes the state. The following list is the meaning of all state names, data types, and data representations.
In the Type column, 32u represents a 32-bit unsigned integer, and "64u" represents a 64-bit unsigned integer, and "32u:32u" represents two 32-bit unsigned integers separated by colons.

Name	Type	Meaning
Pid	523	Server process ID
Uptime	523	Server run time, per second
Time	523	Server Current Unix Time
Version	String	Version number of the server
Rusage_user	32u:32u	User time accumulated by the process (seconds: subtle)
Rusage_system	32u:32u	The cumulative system time for this process (seconds: subtle)
Curr_items	523	The amount of content currently stored by the server
Total_items	523	The total number of content that has been stored since the server started
bytes	64u	The number of bytes consumed by the server currently storing content
Curr_connections	523	Number of connections
Total_connections	523	Total number of connections accepted since the server was running
Connection_structures	523	Number of connection structures allocated by the server
Cmd_get	523	Total Fetch Requests
Cmd_set	523	Total Storage Requests
Get_hits	523	Total number of Successful requests
Get_misses	523	Total number of requests failed
Bytes_read	447	The total number of bytes that the server reads from the network
Bytes_written	447	Total number of bytes sent to the network by the server
Limit_maxbytes	523	Total number of bytes that the server is allowed to use when it is stored

Other commands
The "Flush_all" command has an optional numeric parameter. It always executes successfully and the server sends a "ok\r\n" response. the effect is to invalidate an existing project immediately (the default ), or after a specified time. After that the FETCH command is executed and nothing is returned (unless the same key name is re-stored). Flush_all actually does not immediately release the memory occupied by the project, but is executed when a new project is subsequently stored. The Flush_all effect is as follows:

It causes all items to be updated earlier than the time set by Flush_all, and the command is ignored when the FETCH command is executed .

The "version" command has no parameters:
version\r\n
In the response, the server sends:
"VERSION <version>\r\n"
<version> is the version string of the server.

The "Quit" command has no parameters:
quit\r\n
After receiving this command, the server closes the connection. However, the client can simply close the connection when it is no longer needed, and does not necessarily need to send this command.

UDP protocol
UDP-based interfaces can be used when the number of connections from the client is much larger than the upper limit of the TCP connection . The UDP interface is not guaranteed to be in place, so it is only used in operations that do not require success, such as when used for a "get" request, where errors or responses are lost due to improper cache processing.

Each UDP packet contains a simple frame header, and the content after the data is similar to the description of the TCP protocol. In the execution of the resulting data flow, the request must be contained in a separate UDP packet, but the response may span multiple packets. (only "Get" and "set" request exceptions, spanning multiple packets)

The frame header is 8 bytes long, as follows (all consist of 16-bit integers, network byte order, high in front):

• 0-1 Request ID
• 2-3 Serial Number
• 4-5 total number of packets for this information
• 6-7 reserved bit, must be 0

The request ID is provided by the client. In general, it will be an increment from the random cardinality, but the client wants to use what kind of request ID. The server's response will contain the same ID as the one in the request. The client uses the request ID to differentiate each response. Any packet without a request ID may be caused by a delay in the previous request and should be discarded. The return of the ordinal is the number of packets from 0 to n-1,n that is the message.

Format of the stored command:

12	<command name> <key> <flags> <exptime> <bytes><data block>

The parameters are described as follows:

<command name>	Set/add/replace
<key>	Find keywords
<flags>	The client uses it to store extra information about key-value pairs
<exptime>	The survival time of this data, 0 means forever
<bytes>	Number of bytes stored
<data block>	Stored data blocks (can be directly understood as value in the key-value structure)

1. Add

(1), the set that is stored anyway

This set command is used very frequently in memcached. The set command can not only be added simply, if the set key already exists, the command can update the key corresponding to the original data, that is, to achieve the role of the update.

You can view the added records in the form of "Get Key Name":

As you know, we can also remove the Delete command and add it again.

(2), add only when data does not exist

(3) Replace when the data is present

2. Delete

As you can see, deleting a key value that already exists and a record that does not exist can return different results.

Second, read command 1, get

The key of the GET command can represent one or more keys, separated by a space

2, gets

As you can see, the GET command returns a number (medium 13) More than the normal gets command. This number checks to see if the data has changed. When the data corresponding to the key changes, the number returned will also change.

3. CAs

CAS is the meaning of checked and set, which can only be stored if the last parameter matches the parameters obtained by the GET, otherwise "EXISTS" is returned.

Iii. Status Command 1, stats

2. Stats items

Perform stats items, you can see the stat items row, and if memcached stores a lot of content, there will also be a lot of stat items listed here.

3, stats cachedump slab_id limit_num

We perform the stats cachedump 1 0 command effect as follows:

The slab_id here is 1, which is determined by the results returned by the stats items in 2 (the number following the STAT items); Limit_num looks like it's going to return as many records as it's worth, but 0 means it shows all the records, and N (n>0) Indicates that N records are displayed, and if n exceeds all records under the slab, the result is the same as the result returned by 0.

Limit_num records can be traversed by stats items, stats cachedump slab_id memcached with the Get command.

4. Other Stats commands

such as stats slabs,stats sizes,stats Reset, etc. use is also more common.

Iv. Other common commands 1, append

Add the cached data after the existing cache data, such as the existing cache key does not exist for the server response as not_stored.

2, Prepend

is very similar to append, but it is useful for adding cached data before existing cache data.

3, Flush_all

The command has an optional numeric parameter. It always executes successfully and the server sends a "ok\r\n" response. The effect is to invalidate an existing project immediately (the default), or after a specified time. After that the FETCH command is executed and nothing is returned (unless the same key name is re-stored). Flush_all actually does not immediately release the memory occupied by the project, but is executed later when new items are stored (this is determined by memcached's lazy detection and deletion mechanism).

The Flush_all effect is that it causes all items to be updated earlier than the time set by Flush_all, and the command is ignored when the FETCH command is executed.

Memcached Protocol resolution