Redis cache you must know!

Whether you are engaged in Python, Java, go, PHP, Ruby, etc... Redis should be a relatively familiar middleware. Most programmers who often write business code may only use the set value and get value operations in their actual work. There is a lack of overall understanding of redis. Let's summarize the common problems of redis today. Hope to help you

What is redis?

Redis is an open-source key-value storage database written in C language. It can be used in cache, event publishing and subscription, and high-speed queue scenarios. It also supports a wide range of data types: String, hash, list, set, and zset)

Application scenarios of redis in Projects 1. cache data

The most common data is hotspot data.

2. Message Queue

It is equivalent to a message subscription system, such as activemq and rocketmq. We recommend that you use MQ if you have high data consistency requirements)

3. counters

For example, calculating the click rate and like rate. redis is atomic and can avoid concurrency problems.

4. e-commerce website information

Large e-commerce platforms initialize the cache of page data. For example, when you purchase a ticket online, the homepage price is different from the price you clicked in.

5. hotspot data

For example, the real-time hot spots of news websites and Weibo hot searches need to be updated frequently. When the total data volume is large, directly querying from the database will affect the performance.

Give a reason for love. We usually do this on a single-node server.

Insert image description here
With the development of enterprises, business expansion. In the face of massive data volumes, direct use of MySQL will lead to performance degradation and slow Data Reading and writing. So we can use the cache to process massive data.

 

So now we are like this:

Insert image description here

It only briefly describes the role of the cache. When the data continues to increase, we need to use the master-slave replication technology to achieve read/write splitting.

The database layer interacts directly with the cache. If any data in the cache is directly returned to the client, it will be queried from mysql if no data is returned. This reduces the pressure on the database and improves efficiency.

A new mobile phone is usually released, and there will be a flash sale. During the same time period, the server receives many order requests.

We need to use redis atomic operations to achieve this "single thread ". First, we put the inventory in a list. If there are 10 pieces of inventory, we will put the number of push10 In the list. This number has no practical significance, but only represents 10 pieces of inventory. After the flash sales starts, each user will pop a number from the list, indicating that the flash sales are successful. If the list is empty, it indicates that it has been destroyed. Because the pop operations on the list are atomic, even if many users arrive at the same time, they are executed in sequence.

Digression: another flash sale is to restrict requests directly on the front-end page. These requests are directly intercepted by the front-end and are not directed to the backend server.

Why is redis so fast?

1. redis is a pure memory operation. When needed, we need to manually persist it to the hard disk.

2. redis is a single thread, avoiding frequent context switching in multiple threads.

3. Simple redis data structure and simple data operations

4. Different underlying models are used, and the underlying implementation methods and application protocols for communication between them and clients are different. redis directly builds its own VM mechanism because the general system calls system functions, it will waste some time to move and request

5. Multi-Channel I/O multiplexing model, non-blocking I/O

Multiplexing: I/O multiplexing is a technology that occurs to solve the problem of process or thread blocking to an I/O system call. It can monitor multiple descriptors, once a descriptor is ready (generally read or write, before the file descriptor is read/write), the program can be notified to perform corresponding read/write operations.

Application scenarios of redis Data Types

We mentioned above that apsaradb for redis supports five rich data types. How can we choose between different scenarios?

String

A string is the most common data type. It can store any type of string, including binary, JSON objects, and even base64-encoded images. The maximum size of a string in redis is 512 MB, which can be said to be omnipotent.

Hash

It is often used to store structured data. For example, in a forum system, it can be used to store user ID, nickname, profile picture, points, and other information. If you need to modify the information, you only need to retrieve the value through the key for deserialization and modify the value of a specific item, and then serialize and store the value in redis. The Hash structure is stored, because the hash structure compresses and stores a single hash element when there are not a certain number of elements, it can save a lot of memory. This does not exist in the string structure.

List

The implementation of list is a two-way linked list, that is, it can support reverse lookup and traversal, which is more convenient to operate, but it brings some additional memory overhead, many implementations inside redis, this data structure is also used, including the sending Buffer Queue. In addition, the lrange command can be used to implement redis-based paging functions, with excellent performance and good user experience.

Set

The functions provided by the set interface are similar to the list function. The special feature is that the Set interface can automatically record duplicates. When you need to store a list data and do not want repeated data, in this case, you can select set.

Sort set

You can sort by the weight of a condition, for example, you can use the number of clicks to make a ranking data application.

Redis cache data consistency

In a real sense, the data in the database and the cached data cannot be consistent. The data can be divided into two types: Always and strong consistency. If the data requirements in the business must be strong, the cache cannot be used. What the cache can do is ensure the final consistency of data.

What we can do is to ensure data consistency as much as possible. Data inconsistency may occur between deleting a database, deleting a cache, and then deleting a database. Because read and write operations are concurrent, we cannot guarantee their order. The specific response policy should be determined based on the business needs. We will not go into details here.

Redis expiration and memory Elimination

When redis stores data, we can set its expiration time. But how does one Delete this key?

At first, I thought it was timed deletion, but later I found that this was not the case, because if it was timed deletion, a timer was required to continuously monitor the key. Although the memory was released, however, CPU resources are greatly consumed.

Apsaradb for redis uses Regular deletion. The default value is one check every ms, and an expired key is deleted. The check here is not a sequential check, but a random check. So will there be a fish in the dark? Obviously, redis also takes this into consideration. When we read/write an expired key, redis's inert deletion policy will be triggered and expired keys will be killed directly.

Memory elimination means that some keys stored by users can be automatically deleted by redis, so that data cannot be found from the cache. The memory added to our server is 2 GB, but the cached data has exceeded 2 GB as the business grows. However, this does not affect the running of our program, because the visible memory of the operating system is not limited by the physical memory. It doesn't matter if the physical memory is not enough. The computer will draw a space from the hard disk as the virtual memory. This is the original intention of redis to design two application scenarios: cache and persistent storage.

Cache breakdown

The cache is only a layer of protection layer added to relieve the pressure on the database. When the data we need cannot be queried from the cache, We need to query it in the database. If hackers exploit this vulnerability to frequently access data that is not in the cache, the cache will lose its meaning and the pressure on all requests will immediately fall into the database, this will cause database connection exceptions.

Solution:

1. Set scheduled tasks in the background to actively update cache data. This solution is easy to understand, but the operations are complicated when keys are scattered.

2. Hierarchical Cache. For example, you can set a two-layer cache protection layer. The one-level cache has a short expiration time and the two-level cache has a long expiration time. Some requests are preferentially searched from the level 1 cache. If no corresponding data is found in the Level 1 cache, the thread is locked and the thread obtains data from the database, update cache to level 1 and level 2. Other threads are obtained directly from Level 2 threads.

3. Provides an interception mechanism to internally maintain a series of valid key values. If the requested key is invalid, it is returned directly.

Cache avalanche

Cache avalanche refers to the overall crash of the cache due to some reasons (such as downtime, cache service failure or no response), resulting in a large number of requests arriving at the backend database, leading to database crash, the entire system crashes and a disaster occurs, that is, the cache breakdown mentioned above.
Image Source Network

Insert image description here

How to Avoid avalanche:

1. Add a random expiration time for the cache within a certain range, and set different expiration times for different keys to avoid collective invalidation at the same time.

2. Similar to the cache breakdown solution, the cache is used as a secondary cache to read data from the copy cache when the original cache fails.

3. Use locking or queuing to avoid excessive requests and perform read/write operations on the server at the same time.

Conclusion

Redis has extremely high performance. It reads 110000 times/s and writes 81000 times/S. It supports transactions, backups, and a wide range of data types.

Everything has two sides, and redis also has disadvantages:

1. Because it is a memory database, the amount of data stored on a single machine is limited. developers need to estimate in advance and delete unnecessary data in a timely manner.

2. After modifying the redis data, you need to re-Add the data that is persistent to the hard disk to the content. It takes a long time for redis to run normally.

A circle dedicated to programmer groups, focusing on sharing daily study summaries, industry news, and high-quality learning video resources. Here there are not only technologies, but also poetry and distance... We have prepared a gift for our new friends, including but not limited to Java, Python, Linux, databases, big data, architecture, and e-books in various fields. Reply to the [gift pack] in the public account to receive the package. Long press the QR code to join us for growth ~

Insert image description here

Redis cache you must know!