Memcache Technology Sharing: Introduction, use, storage, algorithm, optimization, hit ratio

Original address: http://zhihuzeye.com/archives/2361

1 , Memcached Introduction

1.1 memcached What is it?

Memcached is a software developed by Brad Fitzpatric, a Danga Interactive company in LiveJournal. It has become an important factor in improving Web application extensibility in many services such as Mixi, Hatena, Facebook, Vox, and LiveJournal. Many Web applications save data to an RDBMS, where the application server reads the data and displays it in the browser. However, with the increase of data volume and the concentration of access, the burden of RDBMS, database response deterioration, site display delay and other significant impact. This is the time to memcached. Memcached is a high-performance distributed memory cache server. The general purpose is to reduce the number of database accesses by caching database query results to improve the speed and scalability of dynamic Web applications.

Built-in memory storage mode

Research memcached This product begins with its memory model: we know that there are two ways to allocate memory in C + +, pre-allocation and dynamic allocation, obviously, pre-allocating memory will make the program faster, but its disadvantage is that the memory can not be effectively used, and dynamic allocation is efficient use of memory, However, the efficiency of the program will be reduced, memcached memory allocation is based on the above principle, obviously in order to achieve faster speed, sometimes we have to space to change time.

Memcached's high performance stems from a two-stage hash (two-stage hash) structure. Memcached is like a huge hash table that stores a lot of <key,value> pairs. With key, you can store or query arbitrary data. The client can store the data on more than one memcached. When querying the data, the client first calculates the hash value (phase hash) of the key by referencing the node list, and the client sends the request to the selected node and
The memcached node finds the real data (item) and returns it to the client through an internal hashing algorithm (phase two hashes). From the implementation point of view, Memcached is a non-blocking, event-based server program.

To improve performance, the data saved in memcached is stored in Memcached's built-in memory storage space. Because the data exists only in memory, restarting the memcached and restarting the operating system will cause all data to disappear. Additionally, when the content capacity reaches the specified value, the unused cache is automatically deleted based on the LRU (leastrecently used) algorithm. Memcached itself is a server designed for caching, so there is no permanent problem with data

Memcached distributed without communication with each other

Memcached Although it is a "distributed" cache server, there is no distributed functionality on the server side. All

Memcached do not communicate with each other to share information. So, how to distribute it? This depends entirely on the client's implementation.

1.2 memcached Start

Memcached initiated commands are in the bin two directory of the installation directory, such as/home/test/app/memcahced-1.4.2/bin/memcached-p 11222-m 128–d

Some common startup options describe the option description

-P listens on a port that defaults to 11211

-M uses memory size, default 64m

-D as daemon in background boot

-VV starting with very vrebose mode, debugging information and error output to the console

-L listens for addresses that are all accessible by default

-M is used to return an error when memory overflows, to prevent automatic migration of

The alternative is to return an error

-p Pid File exists path, only with the-D parameter is used

-c Maximum number of simultaneous connections, default is 1024

Other options that can be viewed with the –h command

1.3 command-line access memcached

The following assumes that memcached starts with the-p parameter of 11311, and the command operation starts at memcached

Native Telnet connects to the memcached server first

Telnet 127.0.0.1 11311

After Telnet is successful, the following information is probably displayed

Trying 127.0.0.1 ...

Connected to Localhost.localdomain (127.0.0.1).

Escape character is ' ^] '.

Various states (stats)

STAT <name> <value>\r\n

For example: Stats command, the following information is returned:

Statsstat PID 26804STAT Uptime 182783STAT time 1404973716STAT version 1.4.13STAT libevent 2.0.11-stablestat pointer_size 6 4STAT rusage_user 2.320647STAT rusage_system 5.411177STAT curr_connections 34STAT total_connections 558STAT connection_ Structures 37STAT Reserved_fds 20STAT cmd_get 127292STAT cmd_set 60056STAT cmd_flush 145STAT cmd_touch 0STAT get_hits 8381 1STAT get_misses 43481STAT delete_misses 15970STAT delete_hits 11992STAT incr_misses 0STAT incr_hits 0STAT decr_misses 0ST At Decr_hits 0STAT cas_misses 0STAT cas_hits 0STAT cas_badval 0STAT touch_hits 0STAT touch_misses 0STAT auth_cmds 0STAT au Th_errors 0STAT bytes_read 14300156STAT bytes_written 11507140STAT limit_maxbytes 134217728 # Memory size allocated to Memcache (bytes) ST At Accepting_conns 1STAT listen_disabled_num 0STAT threads 4STAT conn_yields 0STAT hash_power_level 16STAT hash_bytes 5242 88STAT hash_is_expanding 0STAT expired_unfetched 16884STAT evicted_unfetched 0STAT bytes 609350 # current server storage items occupies the number of bytes sta T curr_items 4668 # Server whenThe number of items previously stored in stat Total_items 60056STAT Evictions 0 # is allocated to Memcache when the space is filled with the need to delete the old items number, kicked out. STAT reclaimed 27160 #回收再利用, an expired data entry to store new data. END

Store command (set, add, replace)

The client sends a line like this:

<command name> <key> <flags> <exptime> <bytes>\r\n

Such as:

Set Key1 0 5\r\nvalue\r\nadd key2 0 2\r\nreplace key1 0 6\r\nvalue1\r\n

Detailed command instructions can be found in the appendix of the memcached in English and Chinese protocol content

Read command (GET)

The command is as follows: Get <key>*\r\n

-<key>* represents one or more key values, a string separated by a space

Such as:

Get key1value key1 0 7value12

Delete command (delete)

Commands such as: Delete <key> <time>\r\n

<key> is the key name of the content that the client wants the server to delete

-<time> is a unit of time in seconds (or on behalf of Unix time until a moment), during which time the server rejects the "add" and "replace" commands for this key name. At this point the content is placed in the delete queue and can no longer be obtained by "get", nor with the "add" and "replace" commands (but the "set" command is available). Until the specified time, the content is eventually erased from the server's memory

The <time> parameter is optional and defaults to 0 (indicates that the content is cleared immediately and subsequent storage commands are available

such as: Delete Key1

Exit command (Quit)

such as: Quit

2. Understanding memcached Memory Storage

Memcache uses the memory allocation mechanism of the slab allocator: divides allocated memory into blocks of a specific length in a predetermined size to completely resolve memory fragmentation issues.
Memcache storage involves slab,page,chunk three concepts
1. Chunk is a fixed-size memory space, which defaults to 96Byte.
2. The page corresponds to the actual physical space, with 1 page 1M.
3. The same size of chunk is also called slab.

2.1, Slab Allocation mechanism: Defragment memory for reuse

The most recent memcached by default uses a mechanism called slab Allocator to allocate and manage memory. Prior to the advent of this mechanism, the allocation of memory was performed simply by malloc and free for all records. However, this approach can lead to memory fragmentation, aggravating the burden on the operating system memory manager, and in the worst case, cause the operating system to be slower than the memcached process itself. Slab Allocator is the principle of Slab Allocation, which was born to solve the problem, is quite simple. Divide the allocated memory into blocks of various sizes (chunk) and divide the same size blocks into groups (chunk collections).

Also, slab allocator has the purpose of reusing allocated memory. In other words, the allocated memory is not freed, but reused.

Main terms of Slab Allocation

Page: The memory space allocated to slab, which is 1MB by default. After assigning to slab, the slab is divided into chunk according to the size of the.

Chunk: The memory space used to cache records.

Slab Class: Group of chunk of a specific size

2.2 The principle of caching Records in Slab

Memcached the list of idle chunk in slab is saved based on the size of the data received, select the slab,memcached that best fits the data size, select chunk from the list, and

And then cache the data in it

2.3 Slab Allocator the shortcomings

The allocated memory cannot be effectively exploited because it allocates memory of a specific length. For example, by caching 100 bytes of data into a 128-byte chunk, the remaining 28 bytes are wasted

There is no perfect solution for this problem, but the more effective solution is documented in the documentation. That is, if you know in advance the common size of the data sent by the client, or if you only cache data of the same size, you can reduce waste if you use a list of groups that fit the data size. Unfortunately, it is not possible to perform any tuning at this time, only to look forward to future versions. However, we can adjust the difference in the size of the slab class. Next, the growth factor option is described.

2.4 tuning with growth Factor

memcached Specifies the growth Factor factor at startup (via the F option) to control the difference between slab to some extent. The default value is 1.25. However, before this option occurs, this factor was once fixed to 2, called the "Powers of 2" policy.

The following is the verbose output after startup:

Slab class 1:chunk size Perslab 8192slab class 2:chunk size Perslab 4096slab class 3:chunk size Perslab 204 8slab class 4:chunk size 1024x768 Perslab 1024slab class 5:chunk size 2048 Perslab 512slab class 6:chunk size 4096 Perslab 256slab class 7:chunk size 8192 Perslab 128slab class 8:chunk size 16384 Perslab 64slab class 9:chunk size 32768 Persla B 32slab class 10:chunk size 65536 Perslab 16slab class 11:chunk size 131072 Perslab 8slab class 12:chunk size 262144 p Erslab 4slab class 13:chunk size 524288 Perslab 2

As you can see, starting with a 128-byte group, the size of the group increases to twice times the original. The problem with this setting is that the difference between slab is large, and in some cases it is quite a waste of memory. So, to minimize memory waste, append the growth factor option two years ago to look at the current default settings (f=1.25) when the output (space is limited, this is only written to the 10th group):

Slab class 1:chunk size Perslab 11915slab class 2:chunk size Perslab 9362slab class 3:chunk size 144 Perslab 728 1slab class 4:chunk Size 184 Perslab 5698slab class 5:chunk size 232 Perslab 4519slab class 6:chunk size 296 Perslab 35 42slab class 7:chunk Size 376 Perslab 2788slab class 8:chunk size 472 Perslab 2221slab class 9:chunk size 592 Perslab 1 771slab class 10:chunk size 744 Perslab 1409

As can be seen, the gap between groups is much smaller than the factor of 2 o'clock, which is more suitable for caching hundreds of-byte records. From the above output, you may find some calculation errors, which are deliberately set to keep the number of bytes aligned. When introducing memcached into a product or deploying it directly using default values, it is best to recalculate the expected average length of the data and adjust the growth factor to get the most appropriate settings. Memory is a precious resource, and a waste of it is too bad.

Item occupancy Space Calculation
*nsuffix = (uint8_t) snprintf (suffix, max, "%d%d\r\n", flags, nbytes–2);

Returnsizeof (item) + Nkey + *nsuffix + nbytes;
*nsuffix= length of "%d%d\r\n"
If the ITEM_CAS flag is set, there are 8 bytes of data
Full item length is key length + value long + suffix long +item structure size (48 bytes) + 8
item.length=56+key.lenght+value.length+ suffix length
The 32-bit machine item structure is 32 bytes
64-bit machine itme structure is 48 bytes
Memcache storage is limited to the length of the key, PHP and C maximum length is 250

3 , memcached delete mechanism

Memcached is cached and does not need to be persisted to the server, this chapter describes the removal mechanism of memcache

3.1memcached Efficient use of resources in data deduplication

Memcached does not release the allocated memory, and after the record expires, the client can no longer see this record, and its storage space is available.

Lazyexpiration

Memcached internally does not monitor whether the record is out of date, but instead looks at the timestamp of the record at get and checks whether the record is out of date. This technique is called lazy (lazy) expiration. As a result, memcached does not consume CPU time on outdated monitoring

3.2LRU: The principle of effectively deleting data from the cache

1.search->refcount = = 0 && deleted that has expired
2.tries = 50; Try a maximum of 50 LRU queues tail find Search->refcount = = 0 First delete
3. tries = 50; Try a maximum of 50 LRU queues tail find Search->refcount! = 0 Query Time (more than 3 hours) of item first delete

Memcached takes precedence over the space of a record that has timed out, but even so, there is a lack of space when appending a new record, and a space is allocated using the least recently used (LRU) mechanism. As the name implies, this is the mechanism for deleting "least recently used" records. Therefore, when memcached has insufficient memory space (when it cannot get new space from the Slab Class), it searches from records that have not been used recently and allocates its space to new records. From a practical point of view of caching, the model is ideal. In some cases, however, the LRU mechanism can cause trouble. The memcached can be disabled by the "M" parameter at startup, as shown below:

$memcached-m–m 1024

It is important to note at startup that the lowercase "m" option is used to specify the maximum memory size. The default value of 64MB is used if no specific value is specified.

When the "M" parameter is specified, memcached returns an error when the memory is exhausted. In other words, memcached is not a memory, but a cache, so it is recommended to use LRU

4 , memcached distributed algorithm

4.1memcached of Distributed

Memcached, although called a "distributed" cache server, has no "distributed" functionality on the server side. The memcached distributed, is fully implemented by the client library. This distribution is the biggest feature of memcached

memcached What does the distribution mean?

The following assumes that the memcached server has node1~node3 three, the application to save the key named "Tokyo", "Kanagawa", "Chiba", "Saitama", "Gunma" data

First add "Tokyo" to the memcached. When "Tokyo" is passed to the client library, the client-implemented algorithm determines the memcached server that holds the data based on the "key". When the server is selected, it commands it to save "Tokyo" and its values

Similarly, "Kanagawa", "Chiba", "Saitama", "Gunma" are the first to select the server and then save the next to obtain the saved data. The key "Tokyo" To get is also passed to the library. The function library selects the server according to the "key" by the same algorithm as when the data is saved. Using the same algorithm, you can select the same server as you saved, and then send a GET command. As long as the data is not deleted for some reason, the saved value can be obtained.

This allows the memcached to be distributed by saving different keys to different servers. memcached server, the key will be scattered, even if a memcached server fails to connect, will not affect the other cache, the system can continue to run

4.2 remainder distributed algorithm

is to "scatter according to the remainder of the number of servers." The integer hash value of the key is obtained, divided by the number of servers, and the server is selected based on the remaining number.

Disadvantages of the remainder algorithm

The remainder calculation method is simple, the dispersion of the data is very good, but also has its shortcomings. That is, when the server is added or removed, the cost of the cache reorganization is significant. When you add a server, the remainder can change so that you cannot get the same server as you saved, which can affect the cache hit.

4.3Consistent Hashing (consistent hash)

Knowledge Supplement: Hash algorithm, that is, hash function. The binary value of any length is mapped to a shorter fixed-length binary value, and this small binary value is called a hash value. A hash value is a unique and extremely compact numeric representation of a piece of data. If you hash a clear text and even change only one letter of the paragraph, subsequent hashes will produce different values. To find two different inputs that hash the same value, it is not possible to compute, so the hash value of the data can verify the integrity of the data. Typically used for quick find and encryption algorithms. (Common with Md5,sha-1)

A brief description of consistent hashing

Consistent Hashing is as follows: first, the hash value of the memcached Server (node) is calculated and configured on the 0~232 Circle (Continuum). It then uses the same method to find the hash value of the key that stores the data and maps it to the circle. It then searches clockwise from where the data is mapped, saving the data to the first server found. If more than 232 still cannot find the server, it will be saved to the first memcached server.

Add a memcached server from the state. The remainder of the distributed algorithm because the server that holds the key changes dramatically, while affecting the cache hit ratio, but in consistent hashing, only the keys on the first server that increase the location of the server counter-clockwise on continuum are affected.

Consistent Hashing: Add server

Therefore, the consistent Hashing minimizes the redistribution of the keys. Moreover, some consistent Hashing implementation methods also adopt the idea of virtual node. With the general hash function, the distribution of the server map location is very uneven. Therefore, using the idea of a virtual node, assign 100~200 points to each physical node (server) on the continuum. This can suppress uneven distribution and minimize cache redistribution when the server is increasing or decreasing.

The result of testing with the Memcached client function library, described above using the consistent Hashing algorithm, is that the hit rate is calculated by increasing the number of servers (n) and the number of servers (m) that are added to the server:

(1 n/(n+m)) * 100

Store command

<command name> <key> <flags> <exptime> <bytes>\r\n

-<command name> is set, add, or Repalce

-<key> is the key value of the data that is required to be stored by the next client

-<flags> saves any 16-bit unsigned shaping (written in decimal) on the server along with data and send blocks when retrieving content. The client can use it as a "bit field" to store some specific information, which is opaque to the server.

-<exptime> is the end time. If 0, the item never expires (although it may be deleted to make room for other cached items). If it is not 0 (the Unix timestamp or the current time of the second offset), the client cannot obtain the content until the end time is reached.

-<bytes> is the byte length of the subsequent chunk of data, excluding the "\ r \ n" used for the distinction. It can be 0 (followed by an empty data block).

-<data block> is a large segment of the 8-bit data whose length is specified by the <bytes> in the preceding command line.

set means "store this data"

add means "to store this data only if the server * does not hold data for this key value"

replace means "to store this data only when the server * has * reserved the data for this key value"

After sending the command line and data block, the client waits for a reply, and the possible reply is as follows:

-"stored\r\n" indicates success.

-"not_stored\r\n" indicates that the data is not stored, but not because an error occurred. This usually means that the condition of the Add or Replace command is not true, or that the item is already in the delete queue (refer to the "delete" command later).

Retrieve command

Get <key>*\r\n

-<key>* represents one or more key values, a string separated by spaces this line of command, the client waits for 0 or more items, each item receives a line of text, and then follows the data chunk. After all the items have been delivered, the server sends the following string: "end\r\n" to indicate that the response is complete, and the server sends each content in the following form:

VALUE <key> <flags> <bytes>\r\n

<data block>\r\n

-<key> is the name of the key being sent

-<flags> is a token set by the store command

-<bytes> is the length of the subsequent data block, * does not include * its delimiter "\ r \ n"

-<data block> is the data sent

If some key names are sent in the retrieval request, and the server does not return the list of items, this means that the server does not have these key names (either because they have never been stored, or have been deleted, or expire, or have been deleted by the client) to make room for other content.

Delete

Delete <key> <time>\r\n

-<key> is the key name of the content that the client wants the server to delete

-<time> is a unit of time in seconds (or on behalf of Unix time until a moment), during which time the server rejects the "add" and "replace" commands for this key name. At this point the content is placed in the delete queue and can no longer be obtained by "get", nor with the "add" and "replace" commands (but the "set" command is available). Until the specified time, the content is eventually purged from the server's memory. The <time> parameter is optional and defaults to 0 (indicates that the content is cleared immediately and subsequent storage commands are available).

This command has one line of response:-"deleted\r\n" indicates successful execution

-"not_found\r\n" indicates that the content was not found

Increase/decrease

The commands "incr" and "DECR" are used to modify the data when some content needs to be replaced, incremented, or reduced. This data must be a decimal 32-bit unsigned integer new. If it is not, it will be treated as a%. The modified content must exist, and when the "incr"/"DECR" command modifies the nonexistent content, it is not treated as 0, but the operation fails.

The client sends the command line:

INCR <key> <value>\r\n or DECR <key> <value>\r\n

-<key> is the name of the content that the client wants to modify

-<value> is the total number of clients to increase/decrease.

Reply to the following centralized scenario:

-"not_found\r\n" indicates the value of the item content and does not exist.

-<value>\r\n,<value> is increased/decreased.

Note the "DECR" command is underflow: If the client tries to reduce the result less than 0 o'clock, the result will be 0. The "incr" command does not overflow.

State

The command "stats" is used to query the server's running state and other internal data. There are two types of formats. With no parameters:

stats\r\n

This will then output the status, setting values, and documentation. Another format with some parameters:

Stats <args>\r\n

Through <ARGS>, the server returns various internal data. Because of possible changes at any time, this article does not provide the types of parameters and their return data.

Various states

After being subjected to the "stats" command without parameters, the server sends multiple lines of content, as follows:

STAT <name> <value>\r\n

The server uses the following line to terminate the list: end\r\n,,<name> is the name of the state in each row state,<value> the data that makes the state. The following list is the meaning of all state names, data types, and data representations.

In the Type column, 32u represents a 32-bit unsigned integer, and "64u" represents a 64-bit unsigned integer, and "32u:32u" represents two 32-bit unsigned integers separated by colons.

Name	Type	Meaning
Pid	523	Server process ID
Uptime	523	Server run time, per second
Time	523	Server Current Unix Time
Version	String	Version number of the server
Rusage_user	523	User time accumulated by the process (seconds: subtle)
Rusage_system	32u	System time accumulated by the process (seconds: subtle)
Curr_items	32u	The amount of content currently stored by the server
Total_items	32u	The total number of content that has been stored since the server started
bytes	447	The number of bytes consumed by the server currently storing content
Curr_connections	32u	Number of connections
Total_connections	523	Total number of connections accepted since the server was running
Connection_structures	523	Number of connection structures allocated by the server
Cmd_get	523	Total Fetch Requests
Cmd_set	523	Total Storage Requests
Get_hits	523	Total number of Successful requests
Get_misses	32u	Total number of requests failed
Bytes_read	447	The total number of bytes that the server reads from the network
Bytes_written	447	Total number of bytes sent to the network by the server
Limit_maxbytes	32u	Total number of bytes that the server is allowed to use when it is stored

If you do not want to view the Memcache status by entering stats each time, you can view it via the echo "stats" |NC IP port, for example: echo "Stats" | NC 127.0.0.1 9023.

5.Memcache hit rate
Cache Hit Ratio = Get_hits/cmd_get * 100% (Total hit count/total number of requests)

To increase the hit rate of memcached, it is necessary to estimate the size of our value and adjust the memory page size and growth factor appropriately.

The increase in hit rate can be achieved through a variety of scenarios.
First, increase the amount of memory that the service acquires
Second, increasing space utilization, which is actually another way to increase the amount of memory
Third, apply a level to the LRU again
Four, for the overall hit rate, you can take an effective redundancy strategy, reduce the distributed service when a server service jitter occurs

6. Some attention
1. Memcache already allocated memory will no longer be actively cleaned up.
2. Memcache the memory page assigned to a slab can no longer be assigned to another slab.
3. Flush_all cannot reset the layout of memcache allocated memory pages, just to expire all of the item.
4. Memcache Maximum stored item (key+value) size limit is 1M, which is limited by page size 1M
5. Because the memcache distributed is the client program through the hash algorithm to get the key to achieve, different languages may be different hash algorithm, the same client program may also use dissimilar methods, so in the multi-language, multi-module sharing the same set of memcached services, It is important to note that the same hash algorithm is selected on the client
6. You can disable LRU substitution with the-m parameter when starting memcached, and add and set will return failure when memory runs out
7. Memcached starts by specifying the amount of data storage, does not include the memory that it occupies, and the administrative space that is set up to save the data. As a result, it consumes more memory than the memory allocation specified at startup, which requires attention.
8. Memcache storage is limited to the length of the key, PHP and C maximum length is 250

Memcache Technology Sharing: Introduction, use, storage, algorithm, optimization, hit ratio