How to improve cache Hit rate _ large web

Introduction to cache Hit ratio

Hit: The desired data can be obtained directly from the cache.

No hits: You can't get the data you want directly from the cache, you need to query the database again, or perform other actions. This may be because the cache does not exist at all, or the cache has expired.

Generally speaking, the higher the cache hit rate, the higher the benefit of using the cache, the better the performance of the application (the shorter the response time, the higher the throughput), the stronger the ability to resist concurrency.

Thus, in the high concurrency of the Internet system, the cache hit rate is a crucial indicator. How to monitor the cache hit ratio

In memcached, the state command is run to view the status information of the memcached service, where Cmd_get represents the total get number, Get_hits represents the total hit count of Get, and the hit rate = Get_hits/cmd_get.

Of course, we can also through a number of open source Third-party tools to monitor the entire memcached cluster, the display will be more intuitive. More typical include: Zabbix, memadmin, etc.

Figure: Memadmin monitoring statistics on the hit rate of the memcached service

Similarly, in Redis you can run the info command to view the status information of the Redis service, where keyspace_hits is the total number of hits, keyspace_misses the total number of miss, and the hit rate =keyspace_hits/ (keyspace_hits+keyspace_misses).

Open Source tools Redis-star can graphically visualize Redis service-related information, and Zabbix also provides relevant plug-ins to monitor Redis services. Several factors that affect the cache hit rate

In the previous section we mentioned the importance of cache hit ratio, and the following analysis of several factors affecting the cache hit rate. Business scenarios and business requirements

Caching is suitable for "read and write less" business scenarios, and conversely, the use of caching is not really significant, the hit rate will be very low.

Business requirements determine the timeliness requirements that directly affect the expiration time of the cache and the update policy. The lower the timeliness requirement, the better for caching. In the case of the same key and the same number of requests, the longer the cache time, the higher the hit rate.

Most of the business scenarios for Internet applications are well suited to use caching. Cache design (granularity and strategy)

Typically, the smaller the granularity of the cache, the higher the hit rate. Give a practical example to illustrate:

When a single object is cached (for example, individual user information), we need to update the cache or allow the cache to be removed only when the corresponding data for that object has changed. When a collection is cached (for example, all user data), the cache needs to be updated or removed when the corresponding data for any one of the objects changes.

In another case, assuming that other places also need to get the data corresponding to the object (for example, other places also need to get individual user information), if the cache is a single object, you can hit the cache directly, conversely, you cannot hit directly. This is more flexible, and the cache hit rate is higher.

In addition, the cached update/expiration policy also directly affects the cache hit rate. When the data changes, the direct update cache value is higher than the cache (or the cache expires), and of course, the system is more complex. Cache capacity and infrastructure

Cache capacity is limited, it is easy to cause cache invalidation and be eliminated (most of the cache framework or middleware uses the LRU algorithm). At the same time, caching technology selection is also critical, such as the use of built-in local cache is more prone to single machine bottlenecks, and the use of distributed caching is easy to expand. Therefore, the system capacity planning needs to be done, and consider whether it can be extended. In addition, there are differences in the efficiency and stability of different caching frameworks or middleware. Other factors

When a cache node fails, the need to avoid cache failure and minimize the impact is a special case that architects need to consider. The typical practice in the industry is through a consistent hash algorithm, or through node redundancy.

Some friends may have this misunderstanding: since the business requirements of the data timeliness requirements are high, and cache time will affect the cache hit rate, then the system does not use the cache. In fact, this ignores an important factor-concurrency. In general, in the same cache time and key case, the higher the concurrency, the higher the cache returns, even if the cache time is very short.

Ways to improve cache hit rates

From the architect's perspective, you need to apply as much data as possible through caching and avoid caching failures. This is also a test of the architect's ability, needs in the business needs, cache granularity, caching strategy, technology selection and other aspects of the overall consideration and trade-offs. As much as possible focus on high-frequency access and low timeliness requirements of the hot business, through the cache preload (preheating), increase storage capacity, adjust cache granularity, update cache and other means to improve the hit rate.

For time-sensitive (or limited cache space), the content span is very large (or access is very random), and the access is not high applications, the cache hit rate may be very low, may be warm after the cache has not come to be accessed to have expired.

Cache expiration and update policy is also very important, the next chapter continues ...