Redis time-out deletion policy

There are three possible answers to this question, each representing three different deletion strategies:

• Timed Delete: When setting the expiration time of the key, create a timer (timer), let the timer at the expiration time of the key to temporarily, perform the delete operation of the key immediately.

• Lazy Delete: Leave the key out of the box, but each time you get the key from the key space, you check to see if the key gets out of date, and if it expires, the key is returned.

• Regular deletion: Every once in a while, the program checks the database and deletes the expiration key. The algorithm determines how many out-of-date keys to delete and how many databases to check.

Of the three strategies, the first and third are the active deletion policy, and the second is the passive deletion policy.

Timed Delete

The timed delete policy is most friendly to memory: by using a timer, a timed delete policy ensures that the expiration key is deleted as soon as possible and frees the memory that the expired key occupies.

On the other hand, the disadvantage of a timed deletion strategy is that it is the most unfriendly to CPU time: When the expiration key is more, deleting the expired key may take up a significant amount of CPU time, in the case of memory tension but CPU time is very tense, Using CPU time on expired keys that are not related to the current task will undoubtedly have an impact on the response time and throughput of the server.

For example, if a large number of command requests are waiting for the server to process, and the server is currently not missing memory, the server should prioritize CPU time on the command request that handles the client instead of deleting the expiration key.

In addition, creating a timer requires a time event in the Redis server, and the current time event is implemented in an unordered list, where the time complexity of finding an event is O (N)-and it is not efficient to handle a large number of time events.

Therefore, to allow the server to create a large number of timers, so that the timing of the deletion strategy, at this stage is not realistic.

Lazy Delete

The lazy delete policy is the most friendly for CPU time: The program only checks out the key when the key is removed, which ensures that deleting the expired key is done only when it is not, and that the target of the deletion is limited to the currently processed key, which does not take any CPU time to delete the other unrelated expiration keys.

The disadvantage of lazy deletion policy is that it is the most unfriendly to memory: If a key has expired and the key remains in the database, the memory it consumes will not be freed as long as the expiration key is not deleted.

When using lazy delete policies, if there are a lot of expiration keys in the database, and those expired keys are just not being accessed, then they may never be deleted (unless the user executes flushdb manually), and we can even think of this as a memory leak-- Useless garbage data consumes a lot of memory, and the server does not release them on its own, which is certainly not good news for Redis servers that run in a very memory-dependent way.

For example, for some time-related data, such as log, after a certain point in time, access to them will be greatly reduced, or even no longer access, if this kind of outdated data backlog in the database, the user thought the server has automatically deleted them, but actually these keys still exist, And the memory of the key is not released, so the consequences are certainly very serious.

Delete periodically

From the above discussion of timed deletions and lazy deletions, both of these deletions have obvious drawbacks when used in a single use:

• Timed deletion consumes too much CPU time, which affects the response time and throughput of the server.

• Lazy removal wastes too much memory and there is a risk of memory leaks.

A recurring deletion policy is a consolidation and compromise between the first two strategies:

• Periodic deletion of policies deletes the expiration key operation once every time and reduces the effect of the deletion on CPU time by limiting the length and frequency of the delete operation.

• In addition, periodic deletion of policies by periodically deleting expiration keys effectively reduces memory waste due to expired keys.

The difficulty of periodically deleting a policy is to determine how long and how often the delete operation is performed:

• If the delete operation is performed too frequently, or if the execution takes too long, the periodic deletion policy is degraded to a timed delete policy, which consumes too much CPU time on the delete expired key.

• If the delete operation is performed too little, or if the execution time is too short, the periodic deletion of the policy will be the same as the lazy delete policy, where there is a waste of memory.

Therefore, if you adopt a periodic deletion policy, the server must reasonably set the execution time and execution frequency of the delete operation, depending on the situation.

The periodic deletion policy of the expiration key is implemented by the Redis.c/activeexpirecycle function, which is called whenever the Redis server periodically operates Redis.c/servercron function execution, and the Activeexpirecycle function is invoked. It iterates through each database in the server in a specified amount of time, randomly checks the expiration time of a part of the key from the expires dictionary of the database, and removes the expiration key from it.

The entire process can be described in pseudo-code as follows:

#Default number of databases per checkDefault_db_numbers= 16#Default number of keys per database checkDefault_key_numbers= 20#Global variables, recording check progresscurrent_db= 0DefActiveexpirecycle(): #Initialize the number of databases to check #If the number of databases on the server is greater thanDefault_db_numbersto small #Then the number of databases on the server will prevail IfServer.Dbnum<Default_db_numbers:Db_numbers=Server.DbnumElse:Db_numbers=Default_db_numbers#Traverse individual databases ForIInchRange(Db_numbers): #Ifcurrent_dbThe value of the server is equal to the number of databases #This means that the checker has traversed all the databases of the server once #Willcurrent_dbReset to0, start a new round of traversal Ifcurrent_db==Server.Dbnum:current_db= 0 #Get the database currently being processedRedisdb=Server.Db[current_db] #Increasing the database index1, point to the next database to be processedcurrent_db+= 1 #Check Database keys ForJInchRange(Default_key_numbers): #If there is not a key in the database with an expiration time, skip the database IfRedisdb.Expires.Size() == 0: Break #Randomly get a key with an expiration timeKey_with_ttl = redisdb.  Expires.  Get_random_key() # Check if the key is out of date and if it expires delete it if   is_expired(key_with_ttl): delete_key(key_with_ttl) # has reached the time limit to stop processing  if reach_time_limit  (): return                

The working mode of the Activeexpirecycle function can be summarized as follows:

• Each time the function is run, a certain number of random keys are removed from a certain number of databases for checking, and the expiration key is removed.

• The global variable current_db records the progress of the current activeexpirecycle function check and then processes the last progress at the next activeexpirecycle function call. For example, if the current Activeexpirecycle function returns when traversing the 10th database, the next time the Activeexpirecycle function executes, it will start looking for and delete the expiration key from database 11th.

• As the Activeexpirecycle function continues to execute, all databases in the server are checked again, and the function resets the current_db variable to 0, and then begins a new round of checks again.

Redis time-out deletion policy