Simple analysis of Redis cache consumption memory data based on Python project (with detailed procedure)

Https://www.cnblogs.com/liuzhen1995/p/9265383.html

Redis: A high-performance Key-value database. Support data persistence, can save in-memory data on disk, restart can be loaded again to use, provide string, list, set, Zset, hash and other data structure of the storage, and support the backup.

This article is suitable for scenarios where the amount of data that Redis caches in a project grows, and the Redis cache's data consumes more memory, and many of them are likely to be less valuable data. Since Redis is a key-value database, analyzing the data in it is not as intuitive as the MySQL database. At this point, we need to look for tools to analyze what data in the Redis cache is taking up more memory, and to analyze the value of these data stores in the context of the actual project. In order to make specific data deletion of the scheme, to liberate the server side of the valuable memory resources.

The tool that this article needs to use: Rdbtools and MySQL.

Rdbtools: written in Python, you can parse a redis dump.rdb file. In addition, the following tools are available:

(1) Memory reports that generate data across all databases and keys

(2) Convert the dump file to JSON

(3) Comparison of two dump files using standard diff tool

Specific source GitHub Link: https://github.com/sripathikrishnan/redis-rdb-tools/

MySQL: An open-source and relatively lightweight relational database. This article uses Rdbtools to parse out a redis dump.rdb file and generate a memory report *.csv file (PS: The following action file is Result_facelive_ HOT.CSV), then import the file into the MySQL database and finally generate the *.csv file for the desired data analysis results by writing a specific SQL statement script (PS: The file named Redis_key_storage.csv is generated in the PM SQL script)

References:

1. Using code to complete the CSV file import mysql (71436108)

2. Use the Rdbtools tool to parse Redis dump.rdb files and memory usage (http://www.ywnds.com/?p=8441)

3.MySQL must know: Combination query (Union) (1190000007926959)

Simple analysis of Redis cache consumption memory data based on Python project (with detailed procedure)