Redis's persistence-rdb and AOF

Although Redis is a memory-based storage system, it natively supports the persistence of in-memory data and provides two main persistence strategies: The Rdb snapshot and the AoF log describe the two different persistence strategies, respectively: 1. Redis's RDB snapshot Redis support will be Snapshot of the previous data into a data file persistence mechanism, the RDB snapshot.      This approach is very well understood, but how does a continuously written database generate snapshots? Redis uses the copy on write mechanism of the Fork command (private memory non-shared memory).      When a snapshot is generated, the current process is forked out of a child process, and then all data is looped through the child process, and the data is written to an RDB file. We can configure the timing of the RDB snapshot generation through the Redis save instruction, for example, you can configure a snapshot to be generated 100 times within 10 minutes, or you can configure a snapshot to be generated with 1000 writes within 1 hours, or you can implement multiple rules together.      The definitions of these rules are in the Redis configuration file, and you can set the rules at Redis runtime with Redis's config set command, without having to restart Redis.     Configuration in Redis: Save 1 #当900秒内有1条keys数据被改变时, generate RDB save #当300秒内有10条keys数据被改变时, generate RDB Save 60 10000 #当60秒内有10000条keys数据被改变时, generating an RDB will the Rdb file break down? The      redis Rdb file does not break because its write operation is performed in a new process, and when a new Rdb file is generated, the Redis-generated subprocess first writes the data to a temporary file. The temporary files are then renamed to an RDB file through an atomic rename system call, so that the Redis RDB files are always available at any time of failure.       At the same time, Redis's Rdb file is also a part of the Redis master-slave synchronization implementation. The following:       The first time, slave to master synchronization is: Slave to master the synchronization request, master first dump the Rdb file, and then the Rdb file full amount to Slave, The master then forwards the cached write command to Slave, which is completed at the first synchronization.       The second and future synchronization implementations are: Master sends snapshots of variables directly to each slave in real time. However, no matter what causes slave and master to disconnect, the process of repeating the above two steps will be repeated. The master-slave replication of      redis is based on the persistence of memory snapshots, and memory snapshots occur as long as there are slave.       However, we can clearly see that the RDB has its shortcomings, that is, once the redis problem, then the data stored in our Rdb file is not entirely new, from the last Rdb file generation to the Redis outage time of the data are all discarded. In some businesses, this is tolerable, and we recommend that these services be persisted using an RDB, because the cost of opening an RDB is not high. But for other applications that have very high data security requirements that cannot tolerate data loss, the RDB is powerless, so Redis introduces another important persistence mechanism: AOF logs. 2, Redis aof log      AOF log is the full name of append-only file, from the name we can see that it is an append write log file. Unlike the binlog of a general database, the AoF file is a plain, recognizable text, and its content is a Redis standard command. Of course, not all commands sent to Redis are recorded in the AoF log, and only those commands that cause data changes will be appended to the AoF file.       each command that modifies the data generates a log, so the aof file is not very large. The answer is yes, the aof file will get bigger and larger, so Redis provides a feature called AoF rewrite (using Redis to provide the bgrewriteaof command). Its function is to regenerate a copy of the AoF file, one record in the new AoF file will only be done once, and unlike the old aof file, multiple operations on the same value may be logged. Its build process is similar to an RDB, and it also fork a process, traversing the data directly, writing a new aof temporary file (this process is similar to an RDB, but it is the form of splitting the data into a single write command). In the process of writing a new file, all of the write logs are still written to the old aof file and are also recorded in the memory buffer. When the write operation is complete, the logs in all buffers are written to the temporary file one time. Then call the atomic Rename command to replace the old aof file with the new AoF file. (Such an operation, the old aof can recover the memory, if the generation of new aof, the old will not exist, the use of new aof files to recover memory, so as to solve the growing problem of aof. ） So how high is the operational security of writing AOF?       Actually this can be set, after calling write write to aof in Redis, when to call Fsync to write it to disk, by Appendfsync option to control, the following Appendfsync three settings, Security intensity is gradually increasing.      1) Appendfsync no       when Appendfsync is set to No, Redis does not actively invoke Fsync to synchronize aof log content to disk. So all this is entirely dependent on the operating system, and for most Linux operating systems, Fsync every 30 seconds, writing data from the buffer to disk.      2) Appendfsync everysec       When setting Appendfsync to Everysec, Redis will default to Fsync calls every second, Writes the data in the buffer to disk. But when the Fsync call is longer than 1 seconds, Redis takes a deferred fsync policy and waits another second. That is, in two seconds after the Fsync, this time Fsync no matter how long it will be carried out. Because the file descriptor is blocked at Fsync, the current write operation is blocked.       The conclusion is that in the vast majority of cases, Redis will fsync once every second. In the worst case, the Fsync operation is performed once in two seconds. This operation is called Group commit in most database systems, which is the combination of multiple write operations and writes the logs to disk at once.      3) Appendfsync always       when the Appendfsync is set, every write is called once Fsync, and the data is the safest, of course, Because Fsync is executed every time, its performance is also affected. 3, both in the data recovery speed comparison      RDB recovery time may be shorter, for two reasons: the Rdb file has only one record per piece of data, unlike the AOF log, there may be more than one record of data operation. So every piece of data just needs to be written once. The format of the Rdb file is consistent with the encoding format of the Redis data in memory and does not require any further data coding, so the CPU consumption is much smaller than the AOF log load.