Batch Delete keys in REDIS

When Redisserverwentaway is used to view system log files, it is found that this error is thrown every day: PHPFatalerror: UncaughtexceptionRedisExceptionwithmessageRedisserverwentaway: the script that throws this problem is a statistical script. You need

When Redis server went away is used to view the system log file, it is found that the error is thrown every day: PHP Fatal error: uncaught exception 'redisexception' with message 'redis server went away 'the script that throws this problem is a statistical script. You need to read the data from the previous day and store the data in the database. At first, it was thought that Redis was reading too frequently.

Redis server went away

When you view the system log file, this error is thrown every day:

PHP Fatal error:  Uncaught exception 'RedisException' with message 'Redis server went away'

The script that throws this problem is a statistical script. You need to read the data from the previous day and store the data in the database. It was initially thought that REDIS was reading too frequently, however, this problem does not occur when the script is executed after the data is imported to the test machine. If the script is executed manually, the Code is not executed. If the script is executed, the execution is very slow. The code for this row is:

$Redis->delete($Redis->keys($pre_key_del.'*'));

A prompt is displayed when you view the manual:

KEYS is very fast, but using it in a large database may still cause performance problems. If you need to find a specific key from a dataset, you 'd better use the set structure of Redis instead.

Log on to redis and view it through info. The memory usage is more than 25 GB, and the KEY also has 0.144 billion... There are a large number of useless keys in the REIDS and no expiration time is set. It is necessary to set an expiration time.

used_memory_human:24.72Gdb0:keys=144856453,expires=25357

Executing keys prefix * on the test machine causes REDIS to become stuck and other connections cannot be connected. Therefore, the problem is located on the keys command, which is also caused by performance problems as described in the Manual.

How do I delete unused keys?

Most keys are regular and have a specific prefix. You need to get the KEY with a specific prefix and delete it. The following command is available on the Internet:

redis-cli -a redis-pwd -n 0 keys "preffix*" | xargs redis-cli -p 6379 -a redis-pwd -n 0 del

Test Machine to run keys "preffix-1 *" time about 40 s, which means that redis to stop 40 s +, and the prefix is set by day, this requires multiple operations, for business reasons, this operation is not allowed, and every minute is money ~
The final method is to first export the key that meets the conditions from the test machine to the text, and the previous statement is obtained through cat text. For example:

redis-cli -p 6380 -a redis-pwd keys "preffix-1*" > /home/keys_redis/preffix-1

Then, the keys in the production environment are deleted using the data.

cat /home/keys_redis/preffix-1 | xargs redis-cli -a redis-pwd -n 0 del

The deletion speed is very fast, and the memory consumption is quite fast. It feels like the amount of memory consumption. After execution, the number of keys is reduced by more than 95%, and the memory is also reduced from 25 GB to 2 GB. However, there is an exponential increase ---- mem_fragmentation_ratio. The memory comparison before and after:

# Memory processing before used_memory: Memory: 25.00Gused _ memory_rss: Memory: 25.11Gused _ Memory: 0.88mem _ allocator: jemalloc-3.2.0 # Memory Processing after used_memory: Memory: 2.23Gused _ memory_rss: 4621533184used_memory_peak: 26963439000used_memory_peak_human: 25.11Gused _ memory_lua: 31744mem_fragmentation_ratio: 1.93mem _ allocator: jemalloc-3.2.0

The mem_fragmentation_ratio problem may need to be optimized. From the redis issue, we can see that when setting the cache, we also need to consider the cache maintenance problem and whether to set the cache expiration time, how to manage the key naming method, you can't just put data into it.

Related Websites:

Redis

Redis command reference

Original address: Delete keys in REDIS in batches. Thanks to the original author for sharing.