Memcached Performance Testing

Memcached Performance Testing

Memcached as a memory Key-value storage container has excellent performance, but in the last use of the discovery of a large number of data loss occurred, resulting in the function of the cache basically disappeared. The specific detection methods are as follows: Test hit ratio

Detection hit rate is the most basic, most macroscopic way, using Telnet to connect to the memcached server, and then execute the stats command to see some macro information, such as.

The more critical properties in this command are Get_hits and get_misses,get_hits, which indicate the number of times the cache hit was read, get_misses is the number of read failures, which is an attempt to read the nonexistent cache data.
hit rate =get_hits/(get_hits + get_misses)
The higher the hit rate, the greater the cache caching effect. But in actual use, this hit rate is not a valid data hit rate, sometimes the get operation may just check that a key exists, this time miss is also correct, it is like using memcached as a timer, Some temporary data in the memcache for a specific length of time, the business logic will be based on the existence of the cache for different logic, this data is not pure cache, and should not be counted in the hit rate. Furthermore, this hit rate is the aggregate value of all requests starting from memcached, which does not reflect the situation within a time period, so to troubleshoot the performance of memcached, more detailed values are required. But the high hit rate still reflects the good usage of the memcached, and the sudden drop in hit rate can reflect the loss of a large number of caches.

Stats Items

The Stats Items command can see some details about the item stored in each slab, as detailed in.

Key attributes are:

The time, in seconds, that the last rejected data is stored in the cache

Stats items can observe the data objects of each slab in detail, because memcached's memory allocation policy causes the total memory of the memcached to reach the maximum memory set, which means that all slab can use the page is fixed. This time if there is data to put in, it will start to cause memcached to use the LRU policy to reject the data. The LRU strategy is not for all slabs, but for the slab that new data should be put into, such as having a new data to be put into slab 3, and LRU only for Slab 3. These culling cases can be observed through the stats items.
The specific analysis is as follows:

Evicted property
If the evicted property of a slab is not 0, then there is a case of early culling of data in slab, and this slab is probably what you need to be aware of. Evicted_time Property
If evicted is not 0, then Evicited_time represents the time of the last data time cache being rejected. It's not that the LRU is overloaded with code memcached load, because sometimes when you use the cache, the expiration time is set to 0, so the cache will be stored for 30 days, and if the memory is slow, it continues to put in the data, and the data that has expired is not used for a long time, it may be rejected. It is important to note that the final culling of this data has been cached time, the Evicted_time converted to standard time to see if it has reached the time you can accept, for example: You think the data is cached for 2 days is acceptable to you, and the last rejected data has been stored for more than 3 days, It can be thought that the slab pressure is acceptable, but if the last data is only cached for 20 seconds, do not consider that the slab is overloaded. Age Property
The age attribute reflects the longest time in the currently cached data, and its size and evicted_time do not have a definite size relationship, because the data that may be the longest time is indeed frequently read and will not be cleaned by LRU, but if it is less than evicted_time, This means that the data has been cleaned up before being read, or cached objects that have been stored for a long time but are not used. Stats Slabs

If you find an unusual slab from the stats items, you can stats slabs to see if the slab is a memory allocation problem.
Stats slabs results such as

The properties of the Stats slabs are described below:

Chunk_size	Current slab size of each chunk
Chunk_per_page	The number of chunk each page can hold
Total_pages	Total number of page assigned to current slab
Total_chunks	The current slab can hold the maximum number of chunk, should be equal to Chunck_per_page * total_page
Used_chunks	Total number of chunks already occupied
Free_chunks	The number of chunk that have not been used in the chunk out of the expired data
Free_chunks_end	Number of chunk that are newly allocated but not yet used

This command has a large amount of information and all properties are valuable. The following one by one explains the properties:

Combining the above data, it can be found that the properties of memcached memory usage decrease are:

Chunk_size, Chunk_per_page
These two properties are fixed, but it reflects the data size of the current slab store, allowing you to analyze the hash interval of the cached data, and adjust the growth factor to change the slab interval distribution, thereby altering the area where the data is hashed. If a large amount of 230byte to 260byte of data, and just one slab size is 250byte, then 250byte to 260byte data will be dropped to the next slab, resulting in a lot of wasted space. Total_pages
This is the total number of page totals that are currently allocated by the slab, and if the default size of the page is not modified, this is the total size (in m) of the data that the current slab can cache. If this slab is very serious, be sure to note that this slab page number is not too small.
The project I worked on last time because of the memcache shared with another project, and Memcache has been running for a long time, causing the page to all be allocated, and the cache data size of just two items is much different, resulting in the most new project data slab 4 unexpectedly only a page, so the data cache less than 22s was replaced, completely lost the meaning of the cache.
For the situation I encountered, the solution was to reassign the page or restart the Memcache service. However, the page reassign method has been completely removed from version 1.2.8, so there is no way to reassign the page online now. The other is sometimes unacceptable, because a reboot of a cache server will cause all cached data to be re-fetched from the DB, which could result in an instantaneous increase in the pressure on the db. And some cache data is not in storage, this time we need to do memcache import and export. In the next article I will summarize the memcache dump operation. Total_chunks
This function is basically the same as total_pages, but this property can more accurately reflect the total number of cached objects that can actually be stored. Used_chunks, Free_chunks, Free_chunks_end
These three attributes are relatively high, and in numerical terms they satisfy:
total_chunks = used_chunks + free_chunks + free_chunks_end
Used_chunks is the literal meaning, the number of chunk that has been used, free_chunks is not all unused chunk, but the number of chunk that have been used but have been recycled because of expiration; Free_chunks_ End is the number of chunk that have never been used in a page.


As can be seen, slab 1 only put an object, but has applied for a whole page, this time used_chunks 1, but Free_chunks is 0, because there is no space for recycling, and free_chunks_end is equal to 10081, It means that so many chunk have never been used. Is the data after the expiration of the stats slabs data, you can find that Free_chunks has a value, is the expiration of the chunk, so 1,used_chunks for the 0,free_chunks_end unchanged.


Why divide two free chunk?
My understanding is this: if Free_chunks_end is not zero, the current slab does not have enough capacity, and if Free_chunks is always 0, it means that a lot of data expiration time is too long or is rejected before it expires, This is to be seen in conjunction with the time (age attribute) of culling data and data retention. So separate statistics these two values can accurately determine the actual idle chunk state, once so chunk was used once, unless the page is re-applied, otherwise free_chunks_end is always 0. So for long-running memcached, perhaps most of this value is 0. Active_slabs, total_malloced
The last two items in the stats slabs output are two statistics, one is the total number of active slab, because slab is numbered, but this number is not necessarily contiguous, because there is a possibility that there are some intermediate intervals where the slab has no value and is not initialized. This will not change the slab number when the slab has a value. So the total number of active slab is not necessarily equal to the maximum number of slab.
Total_malloced This is the total amount of memory that is actually allocated, in bytes, which determines how much memory memcached can actually apply, and if the value has reached the set limit, no new page is assigned. The previously assigned page has also been fixed slab.

Combining the above data, it can be found that the properties of memcached memory usage decrease are:

The difference between the actual size of the data and the chunk in a chunks;chunk that has never been used in a page, because a short period of data is concentrated in a slab area, resulting in a large number of page allocations, and then idle memory, These will not be allocated to the slab area of the actual pressure even if the entire page is idle (does this function memcached be considered for later?). ）。

Transferred from: Http://www.woxihuan.com/53055876/1329751841085858.shtml?isalbum=1

Memcached Performance Testing