Memcache [01]-Communication Protocol

1 -- protocol

The memcached client communicates with the server over TCP (UDP interfaces can also be used. For details, see "UDP ). A given running memcached server listens for connections on a (configurable) port. The client connects to the port, sends commands to the server, reads feedback, and closes the connection.

There is no need to send a special command to end the session. The client can close the connection when it does not need it. Note: We encourage clients to cache their connections to the server, rather than re-establishing connections to the server every time they want to store or read data. Opening multiple connections at the same time in memcache will not significantly affect the performance. This is because memcache is designed to enable many connections (hundreds or thousands of connections when needed) it can also run efficiently. Cache connections can save the time overhead for establishing TCP connections with the server (the overhead for establishing a new connection in the server segment is negligible ).

Memcache communication protocols have two types of data: Text row and unstructured data. A text line is used to send commands from the client to the server and the feedback from the server. Unstructured data is used when the client wants to store or read data. The server strictly and accurately returns the data stored during storage in the form of a response stream. The server does not care about the byte sequence, nor does it know the existence of the byte sequence. Memcahce has no restrictions on Characters in unstructured data and can be any character. When reading data, the client can precisely know the length of the next data block in the text line returned previously.

The text line ends with "\ r \ n. Unstructured data usually ends with "\ r \ n", although \ r, \ n, or any other 8-character can appear in the data block. Therefore, when the client reads data from the server, the length of the data block provided earlier must be used to determine the end of the data stream, second, the end of the data stream is not determined based on "\ r \ n" at the end of the upstream stream, even though the data stream format is actually the same.

2 -- Keyword keys

Memcached uses keywords to differentiate the storage of different data. A keyword is a string that uniquely identifies a piece of data. The length of the current keyword is limited to 250 characters (of course, the client does not need to use such a long keyword); the keyword must not contain control characters and spaces.

3 -- Command commands

Memcahe has three types of commands:
1. Storage commands-(three commands: Set, add, and replace) require the server to install keywords to store data. The client sends a command line and a data block. After the command is executed, the client waits for a line of feedback to indicate whether the command is successfully executed.
2. read command -- (only one command: Get) requires the server to read data based on a set of keywords (a request can always contain one or more keywords ). The client sends a command line containing the request keyword. After the command is passed to the server, the server searches for the data under each keyword, A row of feedback information follows the format of a data block to send data back until the server sends an "end" feedback line.
3. Other commands, such as flush_all and version. These commands do not use unstructured data. For these commands, the client sends a command line of text, waiting for a line of data based on the command characteristics or the last line of "end" to feedback information.

 

All command lines always start with the command name, followed by parameters separated by spaces. Command names are all in lowercase and case sensitive.

4 -- timeout expiration times

Some commands sent to the server include the timeout time (the timeout time corresponds to the data item retention time and the client operation time ). In these examples, the actual sending time is either a Unix timestamp (number of seconds since, January 1, January 1, 1970) or a number of seconds from the current time. In the latter case, the number of seconds cannot exceed 60*60*24*30 (30 days in seconds). If the number of seconds is greater than this value, the server considers this value as a Unix timestamp, rather than the offset of the number of seconds since the current time.

5 -- error message error strings

Each command may be fed back with an error message. These error messages are of the following types:
1. "error \ r \ n"
This means that the client sends a command that does not exist in the Protocol.
2. "client_error \ r \ n"
Indicates that the command line entered by the client has an error. The input does not comply with the Protocol. Is a human-readable error comment.
3. "server_error \ r \ n"
It indicates that the server failed to execute commands due to some errors. It is also a human-readable error comment. In some cases, the server cannot serve the client again due to errors (this rarely happens), and the server closes the connection after an error message occurs. This is the only situation where the server proactively closes the connection to the client.
We will not repeat the error message when running various commands in seconds. When we need to know that the error exists, we should not ignore it. 6 -- storage command storage commands

First, the client generates the following command:
<Command name >#< key >#< flags >#< exptime> # <bytes> \ r \ n
<Data block> \ r \ n

 

Where:
<Command name> is set, add, or replace. Set indicates to store the data. Add indicates to store the data if the server does not save the keyword. Replace indicates to replace the original content when the server already has the keyword.
<Key> is the keyword that the client requires the server to store data.
<Flags> is a 16-bit unsigned integer. The server stores it with the data and returns it together when the data is retrieved. The client may use this value as a bitmap to store special data information. This field is not transparent to the server.
<Exptime>
Is the timeout time. If the value is 0, the data item never times out (but sometimes the data item may be deleted to make room for other data). If the value is not 0, it may be absolute UNIX time, it may also be an offset value from the current time. It ensures that the client will not be able to obtain the data item after the customer segment arrives at this timeout time.
<Bytes> is the number of subsequent data bytes, excluding the Terminator "\ r \ n ". <Bytes> it may be 0, followed by an empty data block.
<Data block> stores data streams.
After sending command lines and data, the client waits for feedback in the following situations:
1. "Stored \ r \ n" indicates that the data is successfully stored.
2. "not_stored \ r \ n" indicates that the sent data is not stored, but this is not because of an error. It occurs when the Add or replace command cannot meet the conditions, or the data item is in the queue to be deleted.
3. error message

7 -- read the retrieval command

The read command is as follows:
Get # <key> * \ r \ n
<Key> * indicates one or more keyword strings separated by spaces.
After the command is sent, the client waits for one or more data items to be returned. The format of each data item is a text line followed by a data block. After all the data items are sent, the server sends the string "End \ r \ n", indicating the end of the server feedback data.
The format of the returned data item is as follows:
Value # <key >#< flags >#< bytes> \ r \ n
<Data block> \ r \ n

 

<Key> is the key word for a data item.
<Flags> indicates the flag field in the client command when the data item is stored.
<Bytes> is the length of the data block that follows the text line, excluding the Terminator "\ r \ n ".
<Data block> is the data part of a data item.
If the data items corresponding to some keywords in the Request command line are not returned, this means that the server does not have the data items marked by this keyword (it may have never been stored, it can also be stored but deleted to free up memory space, or the data item times out, or it is deleted by a client ).

8 -- delete Deletion

The DELETE command allows you to directly delete data items. The command format is as follows:
Delete # <key >#< time> \ r \ n

 

<Key> is the keyword that the client wants the server to delete data items.
<Time> is the number of seconds that the client wants the server to block the ADD and replace commands from using the keyword data item, either relative time or absolute UNIX time. During this period, a data item is put into a delete queue and cannot be read by the GET command. Using add and replace on the data item fails, but the SET command can be used successfully. When this time passes, data items are actually deleted from the server's memory. This parameter is optional. If it does not exist, the default value is 0, which means to delete it from the server immediately.
Server return information:
1. "deleted \ r \ n" indicates that the data item is successfully deleted.
2. "not_found \ r \ n" indicates that the data item specified by this keyword is not found on the server.
3. Other error messages
The following flush_all command invalidates all existing data items immediately.

9 -- increase/decrease increment/Decrement

The "incr" and "decr" commands are used to modify and reduce the content of existing data items. The data is processed as a 32-bit unsigned integer. If the current data is not such data, the content is treated as 0. In addition, the data item that uses the incr/decr command on it must already exist. For a non-existing data item, it is not treated as 0, but ended with an error.
The client sends the command line in the following format:
Incr # <key >#< value> \ r \ n
Or
Decr # <key >#< value> \ r \ n
<Key> is the key word used by the client to modify data items.
<Value> is the number of operations that increase or decrease the number of rows. It is a 32-bit unsigned integer.
The following types of feedback are provided:
1. "not_found \ r \ n"
This indicates that the data item is not found on the server.
2. "\ r \ n"
Value is the new value of the data item after the Add/Remove command is executed.
3. error message.
Note the overflow problem of the "decr" command. If the number of attempts by the client to reduce is smaller than 0, the result is 0. The overflow problem of the "incr" command is not checked. In addition, the length of a data is reduced, but the length of the data returned is not necessarily reduced. This number may be a number with spaces appended, but it is only optimized, so you cannot trust it.

 

10 -- statistics on Statistics

The stats command is used to query the running status of the server and other internal data. It can be distinguished by parameters in two cases:
Stats \ r \ n
Or
Stats # <ARGs> \ r \ n
The first case causes the server to output general statistics and set information and documented content.
In the second case, the server sends various internal data based on specific parameters. This part is not documented in the protocol, because it may change at any time due to the developer of memcache.

 

11 -- general-purpose statistics

After receiving the "stats" command without parameters, the server sends many lines similar to the following format:
Stat # <name >#< value> \ r \ n
After all the similar lines are sent, the server sends the following text lines to end the feedback:
End \ r \ n
<Name> is the name of the statistical item and <value> is its data in all stat text lines. The following is a list of all statistical items reported by the stats command, followed by the Data Type of the value. In the data type column, "32u" indicates a 32-bit unsigned integer, "64u" indicates a 64-bit unsigned integer, "32u: "32u" indicates two 32-bit unsigned integers separated by colons.

Name	Value Type	Description
PID	32u	Process Number of the server process
Uptime	32u	Number of seconds since the server was running
Time	32u	UNIX time on the current server
Version	String	Server version string
Rusage_user	32u: 32u	User time accumulated by server processes (seconds: subtle)
Rusage_system	32u: 32u	System Time accumulated by server processes (seconds: subtle)
Curr_items	32u	Number of data items currently stored on the server
Total_items	32u	Number of data items saved on the server
Bytes	64u	Number of bytes of data stored on the current server
Curr_connections	32u	Number of connections in the enabled status
Total_connections	32u	Number of all connections that have been opened
Connection_structures	32u	Number of connection struct allocated by the server
Pai_get	64u	Number of GET command requests
Performance_set	64u	Number of times of storing command requests
Get_hits	64u	Number of times the keyword is hit
Get_misses	64u	Number of times the keyword is missing
Evictions	64u	Number of data items replaced by memory due to timeout
Bytes_read	64u	Number of bytes read by the server from the network
Bytes_write	64u	Number of bytes written by the server to the network
Limit_maxbytes	64u	Maximum data storage allowed by the server

 

12 -- Other commands other commands

"Flush_all" is a command with optional numeric parameters. Its execution is always successful, and the server always responds with a "OK \ r \ n" string. It is used to make all data items expire immediately (by default) or after a specified timeout period. After a data item is invalidated, NO content will be returned for the READ command, unless the data is stored again after it becomes invalid. Flush_all does not actually release the memory space occupied by these existing data items. The actual usage of the data space occurs when new data items are used to overwrite old data items. The most accurate definition of this command is that it causes all data items whose Update Time is earlier than the time point set by this command to be ignored during retrieval, it is as if it has been deleted.
The purpose of using the flush_all command with a delay is that when you have a memcached server pool and need to refresh all the content, you cannot cleanse all the servers at the same time, in this way, the database pressure may be bumpy because all servers suddenly need to re-establish data content. The latency option allows you to set them to expire every 10 seconds (set the first latency to 0, the second to 10 seconds, and the third to 20 seconds ).
"Version" is a command without parameters. The command format is as follows:
Version \ r \ n
The server sends the following feedback:
1. "version # <version> \ r \ n"
<Version> is the version string returned from the server.
2. error message.
"Verbosity" is a command with numeric parameters. Its execution is always successful, and the server feedback "OK \ r \ n" indicates that the execution is completed. It is used to set the log output detail level.
"Quit" is a command without parameters. The format is as follows:
Quit \ r \ n
When the server receives this command, it closes the connection with this customer. In any case, the client can close the connection at any time without sending the command.

 

13 -- UDP protocol

When the number of TCP-based connections exceeds the upper limit of the TCP connection, we can use UDP protocol instead. However, UDP interfaces do not provide reliable transmission, so they are used for operations that are not strictly required. Typical GET requests may cause loss or incomplete transmission due to cache problems.
Each UDP packet contains a simple frame header, followed by a data stream in the data format described by the TCP protocol. In the current implementation, the request must be included in a single UDP packet, but the returned results may be dispersed in multiple packets. (The only request packet that can be split is a large multi-Keyword GET request and set request, which is more suitable for TCP transmission in view of the Reliability .)
The frame header is 8 bytes long, and the format is as follows (All digits are in the 16-bit network byte order shaping, the top is in the first place ):
0-1 Request ID
2-3 serial number
4-5 Number of data packets contained in the current message
6-7 reserved for later use, currently must be 0
The request ID is provided by the client. A typical value is a value that increases progressively from a random seed. In fact, the client can use any request ID. The server's feedback contains the same request ID as the Request command. The client uses this request ID to differentiate the feedback from the same server. Each data packet containing an unknown Request ID may be caused by delayed feedback. These data packets should be discarded.
The serial number ranges from 0 to n-1, and N indicates the total number of packets in the message. The client sorts the data packets by serial number. The result sequence contains a complete feedback in the same format as the TCP protocol (including the "\ r \ n" summary string ).