Redis Expiration Key Deletion policy

Three Delete policies:

1. Timed Delete: When setting the expiration time of the key, create a timer (timer), let the timer at the expiration time of the key to temporarily, perform the delete operation of the key immediately.

2. Lazy Delete: Leave the key out of the box, but each time you get the key from the key space, the key is checked for expiration, if it expires, the key is deleted, and if it does not expire, the key is returned.

3. Regular deletion: Every once in a while, the program checks the database, deleting the expiration key. The algorithm determines how many out-of-date keys to delete and how many databases to check.

Of the three strategies, the first and third are the active deletion policy, and the second is the passive deletion policy.

The timed delete policy is the most memory-friendly: By using a timer, a timed delete policy ensures that the expiration key is deleted as soon as possible and frees the memory that the expired key occupies. On the other hand, the disadvantage of a timed deletion strategy is that it is the most unfriendly to CPU time: When the expiration key is more, deleting the expired key may take up a significant amount of CPU time, in the case of memory tension but CPU time is very tense, Using CPU time on expired keys that are not related to the current task will undoubtedly have an impact on the response time and throughput of the server.

For example, if a large number of command requests are waiting for the server to process, and the server is currently not missing memory, the server should prioritize CPU time on the command request that handles the client instead of deleting the expiration key.

In addition, creating a timer requires a time event in the Redis server, and the current time event is implemented in an unordered list, where the time complexity of finding an event is O (N)----and it is not efficient to handle a large number of time events.

Therefore, to allow the server to create a large number of timers, so that the timing of the deletion strategy, at this stage is not realistic.

The lazy delete policy is the most friendly for CPU time: The program only checks out the key when the key is removed, which ensures that deleting the expired key is done only when it is not, and that the target of the deletion is limited to the currently processed key, which does not take any CPU time to delete the other unrelated expiration keys.

The disadvantage of lazy deletion policy is that it is the most unfriendly to memory: If a key has expired and the key remains in the database, the memory it occupies will not be freed as long as the expiration key is not deleted.

When using lazy delete policies, if there are a lot of expiration keys in the database that are not yet accessed, they may never be deleted (unless the user executes flushdb manually). We can even look at this as a memory leak----useless garbage data consumes a lot of memory, and the server does not release them on its own, which is certainly not good news for Redis servers that are very dependent on memory for running state.

For example, for some time-related data, such as log, after a certain time, access to them will be greatly reduced, or even no longer access, if such outdated data backlog in the database, the user thought the server has automatically deleted them, but actually these keys still exist, And the memory of the key is not released, so the consequences are certainly very serious.

From the above discussion of timed deletions and lazy deletions, both of these deletions have obvious drawbacks in single use:

1. Timed deletion takes up too much CPU time, which affects the response time and throughput of the server;

2. Lazy removal wastes too much memory, and there is a risk of memory leaks.

A recurring deletion policy is a consolidation and compromise between the first two strategies:

1. Periodically delete the policy to delete the expiration key operation once every time, and reduce the effect of deletion on CPU time by limiting the length and frequency of the deletion operation;

2. In addition, by periodically deleting expiration keys, periodic deletion of policies effectively reduces memory waste due to expired keys.

The difficulty of periodically deleting a policy is to determine how long and how often the delete operation is performed:

1. If the delete operation is performed too frequently, or if the execution takes too long, the periodic deletion policy is degraded to a timed delete policy, which consumes too much CPU time on the delete expired key.

2. If the delete operation is performed too little, or if the execution time is too short, the periodic deletion of the policy will be the same as the lazy delete policy, where there is a waste of memory.

Therefore, if you adopt a periodic deletion policy, the server must reasonably set the execution time and execution frequency of the delete operation, depending on the situation.

The Redis server is actually using both the lazy Delete and the periodic deletion policy: By using these two deletion policies, the server can strike a good balance between the reasonable use of CPU time and the avoidance of wasted memory space.

Redis Expiration Key Deletion policy