Detailed explanation of MemCache and memcache

Detailed explanation of MemCache and memcache

What is MemCache?

MemCache is a free, open-source, high-performance, distributed memory object Cache System for dynamic Web applications to reduce database load. It caches data and objects in the memory to reduce the number of times the database is read, thus improving the speed of Website access. MemCaChe is a HashMap that stores key-value pairs. In memory, it stores the key-value used for any data (such as strings and objects). The data can come from database calls and API calls, or the result of page rendering. The design concept of MemCache is small and powerful. Its Simple Design Promotes Rapid Deployment, easy development, and solves many problems faced with large-scale data caching, the open APIs enable MemCache to be used in most popular programming languages such as Java, C/C ++/C #, Perl, Python, PHP, and Ruby.

In addition, let's talk about the differences between MemCache and MemCached:

1. MemCache is the project name

2. MemCached is the name of the executable file on the MemCache server.

MemCache official website for http://memcached.org/

 

MemCache Access Model

To deepen my understanding, I drew a picture of the MemCache part of the book "Core Principles and case analysis of large-scale website technical architecture", a former Alibaba technical expert Li smart teacher:

Clarify one question in particular,Although MemCache is called "distributed cache", MemCache itself does not have distributed functions., MemCache clusters do not communicate with each other (compared with this, such as JBoss Cache, when Cache data is updated on a server, will notify other machines in the cluster to update the cache or clear the cache data). The so-called "distributed" relies entirely on the implementation of the client program, just like the process shown in the figure above.

Based on this figure, let's take a look at the MemCache write cache process:

1. The application inputs the data to be written to the cache.

2. the API inputs the Key into the routing algorithm module. The routing algorithm obtains a server number based on the Key and MemCache Cluster Server LIST.

3. Obtain the IP address and port number of MemCache from the server number.

4. the API call communication module communicates with the server with the specified number to write data to the server and complete a distributed cache write operation.

The read cache and write cache are the same. As long as the same routing algorithm and server list are used and the application queries the same Key, the MemCache Client Always accesses the same client to read data, the cache hit can be guaranteed as long as the data is cached on the server.

This MemCache cluster approach also takes into account partition error tolerance. If Node2 goes down, the data stored on Node2 will be unavailable, in this case, because Node0 and Node1 still exist in the cluster, the Key value stored in Node2 in the next request is definitely not hit. In this case, the data to be cached is obtained from the database first, then, the routing algorithm module selects a node in Node0 and Node1 based on the Key value, and puts the corresponding data so that the cache can be performed next time. This cluster is a good practice, however, the disadvantage is that the cost is relatively high.

 

Consistent Hash Algorithm

From the figure above, we can see that a very important problem is the management of Server clusters. The routing algorithm is very important, just like the load balancing algorithm, the routing algorithm determines which server to access in the cluster. First, let's look at a simple routing algorithm.

1. remainder Hash

For example, the HashCode of a string 'str' is 50, the number of servers is 3, and the remainder is obtained to 1. str corresponds to node Node1, so the routing algorithm routes 'str' to node1. Because HashCode is random, the remainder Hash routing algorithm ensures that cached data is evenly distributed across the entire MemCache server cluster.

If you do not consider the scalability of the server cluster (for more information about scalability, see the study notes on large-scale website architecture), the remainder Hash algorithm can almost meet the needs of most cache routes, however, it is difficult to scale up a distributed cache cluster.

Assume that the MemCache server cluster is changed from 3 to 4. The server list is changed and the remainder Hash is still used. The remainder of 50 to 4 is 2, corresponding to Node2, however, str originally exists on Node1, which leads to cache miss. If this is not clear enough, you may wish to give an example where the original HashCode is 0 ~ 20 data records of 19, so:

HashCode	0	1	2	3	4	5	6	7	8	9	10	11	12	13	14	15	16	17	18	19
Route to server	0	1	2	0	1	2	0	1	2	0	1	2	0	1	2	0	1	2	0	1

Now I have expanded to four, and the bold and red lines indicate hit:

HashCode	0	1	2	3	4	5	6	7	8	9	10	11	12	13	14	15	16	17	18	19
Route to server	0	1	2	3	0	1	2	3	0	1	2	3	0	1	2	3	0	1	2	3

If I scale up to 20 + sets, only the keys corresponding to the first three hashcodes are hit, that is, 15%. Of course, this is just a simple example. The reality is certainly much more complicated than this, but it is enough to show that the remainder Hash routing algorithm is used, during expansion, a large amount of data cannot be correctly hit (in fact, not only cannot be hit, but the amount of data that cannot be hit still occupies the memory before being removed in the original cache ). This result is obviously unacceptable. In most website businesses, most of the business data-level operation requests are actually obtained through the cache, and only a few read operations will access the database, therefore, the load capacity of the database is designed on the premise of cache. When most of the cached data cannot be correctly read due to server resizing, the pressure on accessing the data falls on the database, which will greatly exceed the load capacity of the database, seriously, the database may be down.

The solution is as follows:

(1) When the website traffic volume is low, usually late at night, the technical team works overtime to resize and restart the server.

(2) Gradually push the cache by simulating requests, so that data in the cache server is redistributed.

2. Consistent Hash Algorithm

The consistent Hash algorithm uses a data structure called the consistent Hash ring to implement Hash ing between keys and the cache server. Let's take a look at my own figure:

The specific algorithm process is: first construct an integer ring with a length of 232 (this ring is called a consistent Hash ring), according to the Hash value of the node name (its distribution is [0,232-1]) place the cache server node on this Hash ring, and then calculate its Hash value (the distribution is also [0,232-1]) based on the Key value of the data to be cached. then, search for the server node whose Hash value is closest to the example Key value clockwise in the Hash ring to complete the ing between the Key and the server.

As shown in the figure, the three Node points are located in three positions on the Hash ring, and the Key value is based on its HashCode. There is a fixed position on the Hash ring, after the position is fixed, the Key searches clockwise for a Node closest to it and stores the data on the Node's MemCache server. What if a node is added with the Hash ring? Let's take a look:

We can see that I have added a Node4 node and only affected the data of a Key value. This Key value should have been stored on the Node1 server. Now I want to go to node4. The adoption of consistent Hash algorithms will indeed affect the entire cluster, but only the rough section will be affected. Compared with the remainder Hash algorithm, this will affect more than half of the impact rate, which is much smaller. More importantly,The more cache server nodes in the cluster, the smaller the impact of node increaseIt is easy to understand. In other words, as the cluster size increases, the probability of hitting the original cache data will become greater and greater. Although a small part of the data cache cannot be read on the server, this proportion is small enough, even if you access the database, it will not cause fatal load pressure on the database.

For specific applications, the 232-length consistent Hash ring is usually implemented using the binary search tree. As for the binary search tree, it is an algorithm problem. You can query relevant information by yourself.

 

Implementation principle of MemCache

First, you must note that the MemCache data is stored inMemoryIn the memory, I personally think it means:

1. Data Access is faster than traditional relational databases, because traditional relational databases such as Oracle and MySQL store data on hard disks to maintain data persistence, and IO operations are slow.

2. MemCache data is stored in the memory, which means that the data will disappear as long as MemCache is restarted.

3. Since MemCache data is stored in the memory, it is bound to be limited by the number of machine digits. This article has been written many times, and 32-bit machines can only use 2 GB of memory space at most, no upper limit for 64-bit machines

Next, let's take a look at the principle of MemCache. The most important thing about MemCache is not the content of memory allocation. The memory allocation method adopted by MemCache is fixed space allocation, or you can draw a picture by yourself:

This image involves four concepts: slab_class, slab, page, and chunk. The relationships between them are:

1. MemCache divides memory space into a group of slab

2. Each slab has several pages. Each page is 1 MB by default. If an slab occupies 100 MB of memory, there should be pages in this slab.

3. Each page contains a group of chunks. The chunk is the place where the data is actually stored. The chunk size in the same slab is fixed.

4. slab with chunks of the same size is organized together and called slab_class

MemCache memory allocation is called allocator. The number of slab instances is limited. The number of slab instances is several, dozens, or dozens. This is related to the configuration of startup parameters.

The value stored in MemCache is determined by the value size. The value is always stored in the slab closest to the chunk size, for example, slab [1] has a chunk of 80 bytes, slab [2] has a chunk of 100 bytes, and slab [3] has a chunk of 128 bytes (The chunk in the adjacent slab basically increases by 1.25. You can use-f to specify this ratio when starting MemCache.), Then a 88-byte value will be placed in slab 2. When placing slab, first, slab needs to apply for memory. The applied memory is in the unit of page. Therefore, no matter the size of the first data, A page with a size of 1 MB is allocated to the slab. After applying for a page, slab splits the page's memory by the chunk size and converts it into a chunk array. Finally, it selects a chunk array for storing data.

What should I do if no chunk can be allocated in this slab? If the MemCache startup does not append-M (LRU is disabled, an Out Of Memory error will be reported if the Memory is insufficient ), then MemCache clears the data in the chunk that is least recently used in this slab, and then puts the latest data. The memory allocation and recovery algorithms for MemCache are summarized as follows:

1. MemCache memory allocation chunk will result in memory waste. The 88-byte value is allocated to the 128-byte chunk (followed by a large usage), and 30 bytes will be lost, however, this avoids the issue of managing memory fragments.

2. The MemCache LRU algorithm is not global and slab-oriented.

3. We should be able to understand why MemCache stores limited value sizes. Because a new data comes in, slab will first apply for a piece of memory in the unit of page, the requested memory is only 1 MB at most, so the value size cannot be greater than 1 MB.

 

Summarize the features and restrictions of MemCache.

The above has made a more detailed explanation of MemCache. Here we will summarize the limitations and features of MemCache again:

1. There is no limit on the data size of items that can be stored in MemCache, as long as the memory is sufficient.

2. The maximum memory used by a single MemCache process on 32-bit machines is 2 GB. This article mentioned this many times, but there is no limit on 64-bit machines.

3. the maximum size of a Key is 250 bytes, which cannot be stored if the length is exceeded.

4. the maximum data size of a single item is 1 MB. Data exceeding 1 MB is not stored.

5. The MemCache server is insecure. For example, if a MemCache node is known, you can directly telnet it and use flush_all to immediately invalidate an existing key-value pair.

6. All items in MemCache cannot be traversed because this operation is relatively slow and will block other operations.

7. The high performance of MemCache comes from the two-phase Hash structure: the first stage is on the client, and a node is calculated based on the Key value through the Hash algorithm. The second stage is on the server, through an internal Hash algorithm, find the real item and return it to the client. From the implementation perspective, MemCache is a non-blocking, event-based server program.

8. When MemCache is set to add a Key value, the input value of expiry is 0, indicating that the Key value is permanently valid. The Key value will also expire after 30 days. For details, see memcache. c source code:

#define REALTIME_MAXDELTA 60*60*24*30static rel_time_t realtime(const time_t exptime) {       if (exptime == 0) return 0;       if (exptime > REALTIME_MAXDELTA) {                                     if (exptime <= process_started)                                               return (rel_time_t)1;                                               return (rel_time_t)(exptime - process_started);         } else {                                                                                return (rel_time_t)(exptime + current_time);            }}

The expiration time is written in the memcache source code. Developers cannot change the expiration time of the MemCache Key value to 30 days.

 

MemCache command Summary

As mentioned above, if you know a node of MemCache and telnet it directly, you can use various commands to operate MemCache. Let's take a look at several MemCache commands:

Life order	For use
Get	Returns the Value corresponding to the Key.
Add	Set a Key value unconditionally. If no Key value is added, the value is overwritten.
Set	Add data according to the corresponding Key value. If the Key already exists, the Operation will fail.
Replace	Replace data according to the corresponding Key value. If the Key value does not exist, the operation fails.
Stats	Return General MemCache statistics (detailed explanation below)
Stats items	Returns the number of items in each slab and the age of the oldest item (the number of seconds from the last visit)
Stats slabs	Return the information of each slab created during MemCache running (detailed explanation below)
Version	Returns the current MemCache version number.
Flush_all	All key values are cleared, but items is not deleted. Therefore, MemCache still occupies memory.
Quit	Close connection

 

Explanation of stats commands

Stats is an important command used to list the status of the current MemCache server. Take a group of data for example:

STAT pid 1023 STAT uptime 21069937 STAT time 1447235954 STAT version 1.4.5STAT pointer_size 64 STAT rusage_user 1167.020934 STAT rusage_system 3346.933170 STAT curr_connections 29 STAT 1_21 STAT connection_structures 49 STAT 1__get 49 STAT 1__set 7458 STAT 1__flush 0 STAT get_hits 7401 STAT get_misses 57 .. (number of hits and misses for delete, incr, decr, and cas, and an additional badval for cas) STAT auth_cmds 0 STAT auth_errors 0 STAT bytes_read 22026555 STAT bytes_written 8930466 STAT limit_maxbytes 4134304000 STAT limit 1 STAT limit 0 STAT threads 4 STAT bytes 151255336 STAT current_items 57146 STAT total_items 580656 STAT evicitions 0

These parameters reflect the basic information of the MemCache server. They mean:

Parameter Name	For use
Pid	MemCache server process id
Uptime	Number of seconds that the server has run
Time	Current UNIX timestamp of the server
Version	MemCache version
Pointer_size	The current operating system pointer size, reflecting the number of digits of the operating system. 64 means that the MemCache server is 64-bit.
Rusage_user	Cumulative user time of the process
Rusage_system	Cumulative system time of processes
Curr_connections	Number of currently opened connections
Total_connections	Number of connections opened after the server is started
Connection_structures	Number of connections allocated by the server
Pai_get	Total get command requests
Performance_set	Total set command requests
Pai_flush	Total number of requests of the flush_all command
Get_hits	The total number of hits is important. The most important parameter in the cache is the cache hit rate, which is expressed by get_hits/(get_hits + get_misses). For example, the cache hit rate is 99.2%.
Get_misses	Total number of missed hits
Auth_cmds	Number of times the authentication command is processed
Auth_errors	Number of failed authentication processes
Bytes_read	Total number of bytes read
Bytes_written	Total number of bytes sent
Limit_maxbytes	Memory size allocated to MemCache (in bytes)
Accepting_conns	Indicates whether the maximum value of the connection has been reached. 1 indicates that the maximum value is reached, and 0 indicates that the maximum value is not reached.
Listen_disabled_num	Count the number of connections that have reached the maximum number of connections on the current server. The number should be 0 or close to 0. If this number keeps increasing, be careful about our service.
Threads	The total number of MemCache threads. Because MemCache threads are based on the event-driven mechanism, no thread corresponds to a user request.
Bytes	Total items bytes stored on the current server
Current_items	Total number of items stored on the current server
Total_items	Total number of items stored after the server is started

 

Explanation of stats slab commands

If you have understood the above MemCache storage mechanism, let's take a look at the information in each slab and take a group of data for example:

 1 STAT1:chunk_size 96 2 ... 3 STAT 2:chunk_size 144 4 STAT 2:chunks_per_page 7281 5 STAT 2:total_pages 7 6 STAT 2:total_chunks 50967 7 STAT 2:used_chunks 45197 8 STAT 2:free_chunks 1 9 STAT 2:free_chunks_end 576910 STAT 2:mem_requested 608463811 STAT 2:get_hits 4808412 STAT 2:cmd_set 5958827113 STAT 2:delete_hits 014 STAT 2:incr_hits 015 STAT 2:decr_hits 016 STAT 2:cas_hits 017 STAT 2:cas_badval 018 ...19 STAT 3:chunk_size 21620 ...

First, we can see that chunk_size (144) of the second slab/chunk_size (96) of the First slab = 1.5, chunk_size (216) of the third slab/chunk_size (144) of the second slab) = 1.5. It can be determined that the growth factor of this MemCache is 1.5, and the chunk_size increases by 1.5 times. Then explain the meaning of the field:

Parameter Name	For use
Chunk_size	Size of each chunk of the current slab, in bytes
Chunks_per_page	The number of chunks that each page can store. Because each page is fixed to 1 MB, that is, 1024*1024 bytes, this value is (1024*1024/chunk_size)
Total_pages	Total number of pages allocated to the current slab
Total_chunks	The maximum number of chunks that slab can store. The value is total_pages * chunks_per_page.
Used_chunks	Number of chunks allocated to storage objects
Free_chunks	Number of chunks used but recycled due to expiration
Free_chunks_end	Number of newly allocated chunks that have not been used. If this value is not 0, it indicates that slab has never experienced insufficient capacity.
Mem_requested	Total number of bytes in the memory space requested to store data in slab. (total_chunks * chunk_size)-mem_requested indicates the number of bytes in slab that are idle, this includes unused slab + Memory waste in slab
Get_hits	Number of get requests hit by slab
Performance_set	Number of all set command requests received in slab
Delete_hits	Number of delete requests in slab
Incr_hits	Number of incr requests in slab
Decr_hits	Number of decr requests hit by slab
Cas_hits	Number of cas requests hit by slab
Cas_badval	Number of cas requests that match but fail to update in slab

We can see that the output of this command is very large, and all information is very useful. For example, there are few chunks used in the first slab and many chunks used in the second slab. In this case, you can increase the MemCache growth factor as appropriate, let a part of the data fall into the first slab, and properly balance the memory in the two slab to avoid space waste.

 

Java Implementation instance of MemCache

After talking about this, as a Java programmer, how can I not write THE IMPLEMENTATION OF THE MemCache client? Many third-party jar packages are provided for the MemCache client, and XMemCached is a good client, XMemCached features high efficiency, non-blocking IO, low resource consumption, support for complete protocols, allow node weight setting, allow dynamic addition and deletion of nodes, support for JMX, support integration with Spring framework, use of connection pools, good scalability and many other advantages, therefore, it is widely used. Here, XMemCache is used to write a simple single MemCache client instance, which has not been verified. It is purely a reference:

Public class MemCacheManager {private static MemCacheManager instance = new MemCacheManager ();/** XMemCache allows developers to adjust the load of MemCache by setting node weights. The higher the weight, the more data the MemCache node stores, the higher the load */private static MemcachedClientBuilder mcb = new XMemcachedClientBuilder (AddrUtil. getAddresses ("127.0.0.1: 11211 127.0.0.2: 11211 127.0.0.3: 11211"), new int [] {1, 3, 5}); private static MemcachedClient mc = null; /** initialize and load the client MemCache letter Information */static {mcb. setCommandFactory (new BinaryCommandFactory (); // use the binary file mcb. setConnectionPoolSize (10); // Number of connection pools, that is, the number of clients. try {mc = mcb. build ();} catch (IOException e) {e. printStackTrace () ;}} private MemCacheManager () {} public MemCacheManager getInstance () {return instance ;}/ ** set data to the MemCache server */public void set (String key, int expiry, Object obj) throws Exception {mc. set (key, expiry, Obj);}/** get data from MemCache server */public Object get (String key) throws Exception {return mc. get (key);}/*** MemCache implements atomic update through the compare and set protocol, that is, the cas protocol. It is similar to the optimistic lock. Each request stores a certain data with a cas value, memCache * compares the cas value with the cas value of the currently stored data. If the cas value is equal, the old data is overwritten. If the cas value is not equal, the update fails, this is particularly useful in the concurrent environment */public boolean update (String key, Integer I) throws Exception {GetsResponse <Integer> result = mc. gets (key); long cas = result. getCas (); // update the corresponding key Value if (! Mc. cas (key, 0, I, cas) {return false;} return true ;}}