redis--Persistence article

As is known to all, Redis is an in-memory database that stores data in memory, which also creates new problems with data security by speeding up the read speed, which means that all data in the Redis database will be lost when the Redis server goes down.

To solve this problem, Redis provides persistent functionality--rdb and aof. In layman's words, the data in memory is written to the hard disk.

First, the full amount of persistent write: RDB

 [[Email protected]6381 ]$ more/usr/local/redis/conf/redis.conf Save  900  1  save  10  save  60  10000  dbfilename   dump.rdb   "  #持久化文件名称dir  "  /data/dbs/redis/6381   #持久化数据文件存放的路径 

Above is the default RDB persistence setting in the Redis configuration file, and the first three rows are a condition that triggers an RDB, for example, in a row that means that every 900 seconds a data is modified in the Redis database to trigger an RDB, and so on. ; The bgsave is called for RDB persistence whenever there is a satisfaction. Row Dbfilename Specifies the name of the local file to which the in-memory database is written, which is a compressed binary file that can be used to restore the database to the state of the database when the file was generated. Line five dir specifies the directory where the Rdb file is stored.

Configuration file modification requires a restart of the Redis service, and we can configure it on the command line, immediately, and reconfigure after the server restarts.

[[email protected] redis]$ bin/redis-CLI127.0. 0.1:6379> CONFIG GET save         #查看redis持久化配置 1"save"  2"1 10000"127.0. 0.1:6379"21600"  #修改redis持久化配置OK

There are two types of RDB persistence: Save and Bgsave

Save is a blocking RDB persistence, and when this command is executed, the master process of Redis writes the in-memory database state to the Rdb file (that is, the dump.rdb above), until the time the file is created, Redis will not be able to process any command requests.

Bgsave is a non-blocking persistence that creates a child process that specifically writes the in-memory database state to the Rdb file, while the master process can also handle command requests from the client. However, the child process is basically the parent process of replication, which is equal to two redis processes of the same size running on the system, resulting in a significant increase in memory utilization.

(I encountered this problem in the production, Redis itself memory utilization of 60%, the total memory utilization in 70% or 80% to do encounter, persistent when the time to soar to more than 130%, the alarm message is dozens of per day/(ㄒoㄒ)/~~ finally according to the requirements of the choice of aof persistence)

Two, persistent incremental write: AOF

Unlike the RDB's preservation of the entire Redis database state, AOF records the database state by saving write commands (such as set, Sadd, Rpush) on the Redis server, saving your write to the Redis database, and the following is the contents of the AoF file

1[[Email protected]]$ more appendonly.aof2*23$64 SELECT5$16 07*38$39 SETTen$ - One Dev_user_legal_f9683be0e27f1a06c0cb869cec7e3b22 A$ One - ¬ -*3 the$3 - SET -$ -

First let's see How to configure AoF

1[Email protected]]$ more ~/redis/conf/redis.conf2Dir"/data/dbs/redis/6381"#AOF文件存放目录3 appendonly Yes #开启AOF持久化, default off4Appendfilename"appendonly.aof"#AOF文件名称 (default)5 Appendfsync No #AOF持久化策略6Auto-aof-rewrite-percentage -#触发AOF文件重写的条件 (default)7Auto-aof-rewrite-min-size 64MB #触发AOF文件重写的条件 (default)

To understand that several of the above configurations have to be understood from the AOF implementation, the persistence of aof is achieved through three steps of command append, file write, and file synchronization. When Reids turns on aof, the server appends the command to the end of a separate aof buffer for each write operation (such as set, Sadd, Rpush), which is the command append, and then writes the contents of the AOF buffer to the aof file. Looks like the second step is done. AoF persistence What's the third step? This needs to start from the system's file-writing mechanism: Generally we now use the operating system, in order to improve the writing efficiency of the file, there will be a write policy, that is, when you write data to the hard disk, the operating system is not real-time to write data to the hard disk, but first to temporarily save the data in a memory buffer, When the memory buffer space is filled up or exceeds the set time limit, the data in the buffer is actually written to the hard disk. In other words, when Redis is written to the second step, the data in the AOF buffer is written to the AoF file from the user's point of view, but the system simply puts the contents of the AOF buffer in the other memory buffer. Redis will then need to do file synchronization to actually write the data in the memory buffer to the hard disk to be considered a persistent one. When synchronizing files is done according to the configured Appendfsync:

Appendfsync has three options: Always, everysec and no:

1, choose always when the server will be executed in each event on the AOF buffer content is forced to write to the aof file on the hard disk, you can think of every Redis write command to the AoF file to record this command, which ensures the integrity of data persistence, but the efficiency is the slowest, but also the safest;

2, CONFIGURED to everysec words each time the server performs a write operation (such as set, Sadd, Rpush) also appends the command to the end of a separate aof buffer and writes the AOF buffer to the aof file. And then once every second, the file synchronization will be in the memory buffer in the AOF cache data is actually written into the aof file, this mode takes into account the efficiency and ensure the integrity of the data, even if the server down will only lose a second of the Redis database modifications;

3, the Appendfsync configuration to no means that the data in the Redis database even if the loss you can accept, it will also append each write command to the end of the aof buffer, and then write to the file, but when the file synchronization really writes data to the AoF file is determined by the system itself, That is, the system syncs automatically when the memory buffer space is filled or if it exceeds the set time limit. This mode of efficiency is the fastest, but the data is also the most insecure, if the Redis data are from the back-end database such as MySQL taken out, belong to the CIS can be retrieved or unimportant data, then you can consider the set up and mode.

AOF persistence is doubled in memory each time the RDB is persisted, except that a new child process is created on the first start to create a aof file that consumes memory significantly, and each subsequent persistence is small for memory usage. But AOF also has a problem that cannot be ignored: The aof file is too large. Every write you make to the Redis database will add a piece of data to the AoF file, which will form a monster over time. Fortunately, Redis presents the AOF rewrite mechanism, which is the auto-aof-rewrite-percentage and auto-aof-rewrite-min-size that we configured above. AOF rewriting mechanism here is not detailed here, after I will open another blog to explain this, interested students can see. All we need to know is that the aof rewrite is both a re-creation of a streamlined aof file that removes redundant redundant commands and overwrites the original aof file. This ensures that the size of the aof file is acceptable to the point.         The above auto-aof-rewrite-percentage and auto-aof-rewrite-min-size configurations trigger the AOF override condition. Redis records the file size of the AoF file after the last rewrite, and the current aof file size exceeds the value of the Auto-aof-rewrite-percentage setting for the aof file size after the last rewrite. At the same time, the current aof file size also exceeds the minimum value set by Auto-aof-rewrite-min-size, which triggers the aof file rewrite. As an example of the above configuration, when the current aof file is larger than 64MB and is larger than the file size of the last rewrite aof, the file will be aof rewritten.

Finally, it is important to note that if Redis turns on the AOF persistence feature, the Redis service will take precedence over the aof file to restore the database when it restarts.

redis--Persistence article