Memcached is an efficient distributed memory cache, understanding the memcached memory management mechanism, so that we can understand the memcached, let us tune the characteristics of our data to make it better for me. Here is a brief talk about my memory management of memcached Some of the knowledge, in the absence of special mention, the memcached here is 1.2 version, 1.1 and 1.2 have some differences.
Basic concepts: slab and chunkThere are two very important concepts in the memcached memory structure: Slab and chunk, we first have a perceptual understanding of these two concepts: Figure 1 memcached Memory Structure Slab is a block of memory, which is the smallest unit of memcached requesting memory at a time. When you start memcached, you typically use the parameter-m to specify its available memory, but not all of the memory is allocated at the moment of startup, and you apply only when you need it, and each application must be a slab. The size of the slab is fixed at 1M (1048576 Byte), and a slab consists of several chunk of equal size. An item struct, a pair of keys, and value are saved in each chunk. Although the size of chunk in the same slab is equal, but chunk in different slab are not necessarily equal, memcached can be divided into many categories (class) in chunk according to the size of slab. When you start memcached, you can see the type of slab by-VV: $ memcached-vv
Slab class 1:chunk Size Perslab 13107
Slab class 2:chunk size 104 Perslab 10082
Slab class 3:chunk size 136 Perslab 7710
Slab class 4:chunk size 176 Perslab 5957
Slab class 5:chunk size 224 Perslab 4681
Slab class 6:chunk size 280 Perslab 3744
Slab class 7:chunk size 352 perslab 2978
Slab class 8:chunk Size Perslab 2383
Slab class 9:chunk Size 552 Perslab 1899
Slab class 10:chunk size 696 Perslab 1506
Slab class 11:chunk size 872 Perslab 1202
Slab class 12:chunk size 1096 Perslab 956
Slab class 13:chunk size 1376 perslab 762
Slab class 14:chunk size 1720 Perslab 609
Slab class 15:chunk size 2152 Perslab 487
Slab class 16:chunk size 2696 Perslab 388
Slab class 17:chunk size 3376 Perslab 310
Slab class 18:chunk size 4224 Perslab 248
Slab class 19:chunk size 5280 Perslab 198
Slab class 20:chunk size 6600 Perslab 158
Slab class 21:chunk size 8256 Perslab 127
Slab class 22:chunk size 10320 Perslab 101
Slab class 23:chunk size 12904 Perslab 81
Slab class 24:chunk size 16136 Perslab 64
Slab class 25:chunk size 20176 Perslab 51
Slab class 26:chunk size 25224 Perslab 41
Slab class 27:chunk size 31536 Perslab 33
Slab class 28:chunk size 39424 perslab 26
Slab class 29:chunk size 49280 perslab 21
Slab class 30:chunk size 61600 perslab 17
Slab class 31:chunk size 77000 perslab 13
Slab class 32:chunk size 96256 perslab 10
Slab class 33:chunk size 120320 Perslab 8
Slab class 34:chunk size 150400 Perslab 6
Slab class 35:chunk size 188000 Perslab 5
Slab class 36:chunk size 235000 Perslab 4
Slab class 37:chunk size 293752 Perslab 3
Slab class 38:chunk size 367192 Perslab 2
Slab class 39:chunk size 458992 Perslab 2
Slab class 40:chunk size 573744 Perslab 1
Slab class 41:chunk size 717184 Perslab 1
Slab class 42:chunk size 1048576 perslab 1 Figure 2 Slab group information from the can see, by default memcached divides slab into 40 classes (CLASS1~CLASS40), in Class 1, The size of the chunk is 80 bytes, because the size of a slab is fixed at 1048576 bytes (1M), so there can be up to 13,107 chunk:13107x80 in Class1 and 16 = 1048576 in Class1, the remaining 16 bytes are not enough for one The size of a chunk (80byte), so it will be wasted. The size of each type of chunk has a certain formula, assuming I represents the classification, Class I is calculated as follows: Chunk size (Class I): (default_size+item_size) *f^ (i-1) + chunk_align_bytes
- Default_size: The default size is 48 bytes, that is, memcached default key+value size is 48 bytes, you can use the-n parameter to adjust its size;
- The length of the ITEM_SIZE:ITEM structure is fixed at 32 bytes. The default_size size is 48 bytes, Item_size is 32, so the chunk size of Class1 is 48+32=80 bytes;
- F is factor, is the factor of chunk change size, the default value is 1.25, the adjustment f can affect the step size of the chunk, can be used at startup to specify the-F;
- Chunk_align_bytes is a correction value that is used to ensure that the size of the CHUNK is an integer multiple of a value (the size of the CHUNK required on a 32-bit machine is an integer multiple of 4).
From the above analysis can be seen, we can actually adjust the parameters have-F,-N, in the actual operation of memcached, we also need to observe our data characteristics, reasonable adjustment of the value of f,n, so that our memory is fully utilized to reduce waste.
Memory Request AllocationMemcached memory management to take pre-allocation, group management, group management is the slab class we mentioned above, according to the size of chunk slab is divided into many kinds. The following explains the memory pre-allocation process for memcached. When adding an item to memcached, Memcached first chooses the most appropriate slab class based on the size of the item: for example, the size of item is 190 bytes, and by default class 4 has a chunk size of 160 bytes, which is obviously inappropriate. The chunk size of Class 5 is 200 bytes, greater than 190 bytes, so the item will be placed in Class 5 (obviously there will be 10 bytes of waste is inevitable), after calculating the chunk to be put in, Memcached will check the size of the chunk also have no idle, if not, will apply for 1M (1 slab) space and divided into the kind of chunk. For example, when we first put a 190-byte item into the memcached, memcached produces a slab Class 2 (also called a page) and uses a chunk, the remaining 5,241 chunk to be used for the next size item. When we run out of all 5,242 chunk, the next time an item is added between the 160~200 bytes, memcached will again produce a Class 5 slab (so there are 2 pages). To view the usage of slab, we can telnet IP port and enter the command stats slabs: For example: Telnet 10.0.4.210 11211
| Stats Slabs Stat 5:chunk_size stat 5:chunks_per_page 5242 stat 5:total_pages 1 Stat 5: Total_chunks 5242 stat 5:used_chunks 5242 stat 5:free_chunks 0 Stat 5:free_chunks_end 5241 Stat active_slabs 1 Stat total_malloced 1048400 |
Figure 3 Stats Slab Figure 3 shows the results of the first time after placing a 190-byte item. Summary: I understand that memcached at the start of the M parameter is the allocation of how much memory. Not assigned on start-up: Instead, the memcached is dynamically allocated when used. When you set an item, memcached will determine the size of the new piece of slab each block is the same according to the size of your item. Instead of the slab of different class classes, the redistribution chunk is not the same ... This is where you can optimize your memcached. Echo Stats sizes based on your application | NC 127.0.0.1 11211 | Less view the size distribution of the memory majority item: Then assign the slab of the same kind to the matter can be adjusted. Can look at this http://blog.sina.com.cn/s/blog_6ab686dc01010169.html about memcached allocating memory, there are a lot of online writing is very good.
memcached Memory Management Analysis (RPM)
