How to increase cache hit ratio

Introduction to cache Hit ratio

Hit: The required data can be obtained directly from the cache.

No hits: You cannot get the desired data directly from the cache, you need to query the database again or perform other operations. This may be because the cache does not exist at all, or the cache has expired.

Generally speaking, the higher the cache hit rate, the higher the benefit of using the cache, the better the performance of the application (the shorter the response time, the higher the throughput), the stronger the ability to resist concurrency.

Thus, in high-concurrency Internet systems, the cache hit rate is a crucial indicator.

How to monitor the cache hit ratio

In memcached, run the state command to view status information for the memcached service, where Cmd_get represents the total get count, Get_hits represents the total hit count for get, and hit ratio = Get_hits/cmd_get.

Of course, we can also monitor the entire memcached cluster through some open source third-party tools, and the display will be more intuitive. More typical include:Zabbix,memadmin and so on.

Memadmin monitoring statistics on the hit rate of memcached services

Similarly, in Redis you can run the info command to view the status information for the Redis service, where keyspace_hits is the total number of hits, keyspace_misses is the total number of miss, the hit rate =keyspace_hits/ (keyspace_hits+keyspace_misses).

Open Source tools Redis-star can graphically visualize Redis service-related information, while Zabbix also provides relevant plug-ins to monitor the Redis service.

Several factors affecting the cache hit ratio

In the previous chapters we mentioned the importance of cache hit ratios, which are analyzed below for several factors that affect cache hit ratios.

1. Business scenarios and business requirements

Caching is suitable for "read and write less" business scenarios, conversely, the use of the cache is not really significant, the hit rate will be very low.

Business requirements determine the timeliness requirements that directly affect the expiration time of the cache and the update strategy. The lower the timeliness requirement, the better it is for caching. In the case of the same key and the same number of requests, the longer the cache time, the higher the hit rate.

Most of the business scenarios for Internet applications are well suited for use in caching.

1. Design of the cache (granularity and policy)

Typically, the smaller the granularity of the cache, the higher the hit rate. To give a practical example:

When a single object is cached (for example: individual user information), we need to update the cache or let the cache be removed only if the data corresponding to that object changes. When you cache a collection (for example, all user data), the data for any one of the objects changes, you need to update or remove the cache.

In another case, assuming that other places also need to obtain the corresponding data of the object (such as other places also need to obtain a single user information), if the cache is a single object, you can directly hit the cache, conversely, you cannot hit directly. This is more flexible and the cache hit rate is higher.

In addition, cached update/expiration policies directly affect the cache hit ratio. When the data changes, updating the cached values directly is higher than removing the cache (or letting the cache expire), and of course, the system is more complex.

1. Cache capacity and infrastructure

Cache capacity is limited, it is easy to cause cache invalidation and elimination (currently most of the cache framework or middleware are using the LRU algorithm). At the same time, the cache technology selection is also very important, such as the application of the built-in local cache is more prone to single-machine bottlenecks, and the use of distributed cache is easy to expand. Therefore, it is necessary to plan the capacity of the system and consider whether it can be expanded. In addition, there are differences in the efficiency and stability of different caching frameworks or middleware.

1. Other factors

When a cache node fails, you need to avoid cache invalidation and minimize the impact, a special case that architects need to consider. The most typical practice in the industry is through a consistent hash algorithm, or through node redundancy.

Some friends may have this misunderstanding: since business requirements are highly data-sensitive, and cache time affects cache hit ratios, the system does not use caching. In fact, this ignores an important factor-concurrency. In general, the higher the concurrency, the higher the benefit of the cache, even if the cache time is short, in the same cache time and key.

Ways to increase cache hit ratio

From an architect's perspective, it is necessary for the application to obtain data directly from the cache as far as possible, and to avoid cache invalidation. This is also a test of the architect's ability, need to be in business requirements, cache granularity, caching strategy, technology selection and other aspects to consider and make a balance. As far as possible focus on high-frequency access and timeliness of low-demand hot business, through the cache preload (preheating), increase storage capacity, adjust cache granularity, update the cache and other means to improve the hit ratio.

For highly time-sensitive (or limited cache space), the content span is very large (or access is random), and access is not high, the cache hit rate may be very low, it is possible that the pre-warmed cache has not been accessed has expired.

The cache invalidation and update policy is also very important, the next article continues ...

How to increase cache hit ratio