Redis Rdb and aof persistent __redis source reading notes

Rdb Persistence and aof persistence first, the RDB of persistence

The

Redis is a memory database, the database state is saved in memory, and the server process exits the database state. Redis provides RDB persistence to save the database state, which is the key value pair of the database, to disk. RDB persistence can be performed manually through the Save and Bgsave commands, or it can be performed periodically through a configuration file. The Rdb file generated by the
Rdb persistence feature is a compressed binary file that redis the server to restore the state of the database by loading the Rdb file. RDB file creation and loading: the Save and Bgsave commands generate the Rdb file, and the
a) Save command blocks the Redis server process until the Rdb file is created, and the server cannot process client command requests during blocking. The
B) bgsave Command fork A child process that generates RDB files in the background, and the main process handles client command requests. However, the Save,bgsave and bgrewriteaof commands are rejected.
C) File loading: Redis detects the presence of a RDB file when the server is started and automatically loads the Rdb file. However, because the aof file is newer than the Rdb file, so if the aof persistence function is turned on, use the AoF file to restore the database state, and if not, use the Rdb file. The server will always be blocked when loading the Rdb file until the load is complete. Automatic interval Save: By setting the Save option in the configuration file, the server automatically executes the Bgsave command every other time, such as: Save 900 1 means that the Bgsave command is executed at least once in 900 seconds for the database to be modified.
a) How to automate the retention of intervals: (savaparams+dirty+lastsave+ Servercron 1. Load profile information when the server is turned on, save multiple save options in an array of saveparams in the redisserver structure, which is an array of record-saving conditions. 2, the server state Redisserver maintains the counter record how many times the database state has been modified since the last persisted Rdb file, and maintains the Lastsave property, recording the last time the Rdb file was successfully persisted. 3. Check whether two conditions are met through the Servercron function, as long as the number of changes is greater than the set and the last save time is greater than the set time to execute the bgsave command to persist the database state. two, aof persistence

RDB Persistence records the state of a database by saving a key-value pair in the database, AOF persistence saves the database state by saving write commands executed by the Redis server. All write commands are saved in a command request protocol format, which is a plain text format. The Redis server can restore the database state by loading the execution aof file when it starts.

First, aof persistence implementation: Divided into command append, file write and file synchronization.

A) command append: When the aof persistence is turned on, the server executes a write command that stores the write command at the end of the aof_buf buffer in the redisserver structure, which is SDS.
b Command write: Redis server is an event driver, the first step in an event loop is File Event processing command request and command reply, then time event, general call Servercron function to maintain service side state, Call Flushappendonlyfile before the end of the event loop to write and synchronize the write command in the AOF_BUF buffer to the aof file.
c) file synchronization: Inside the operating system, memory read-write speed is much higher than the disk read and write speed, call write function to disk file writes data will temporarily save the written file in a memory buffer, such as buffer full or time to the disk file. The previous function to write commands to the aof file Flushappendonlyfile, because the configuration in the configuration file will behave differently. If the configuration always indicates that the file is synchronized immediately when writing to the AoF file. Everysecond indicates that files are synchronized once every second. No indicates that writing to the AoF file is not synchronized and when the operating system decides to synchronize.
AOF file loading and data restore: The server creates a pseudo client (the Redis command can only be implemented in the client context), parsing reads a write command from the AoF file, executes a write command using a pseudo client, and then loops through the Write command until the file executes.
Ii. AoF Overrides (sub-process background rewrite +aof rewrite buffer):
Write commands increasing, aof file volume expansion, resulting in the server hard disk space pressure increased, and data restore time increased. By traversing the key space of the database, ignoring the expiration key, using a write command to record the storage state of these keys instead of multiple commands, this is the implementation principle of aof rewrite.
1. AOF background rewrite (subprocess): Call the Aof_rewrite function to generate a new aof file, but the Aof_rewrite function will have a large number of writes, Redis single-threaded model, the thread that calls the function is blocked from processing the client's command request for a long time. So redis the aof rewrite into the subprocess execution. The Redis server fork a subprocess, the subprocess performs the aof rewrite, and the main process continues processing the client command request.
2. After the child process is created, the server uses a aof rewrite buffer, and the write command from the client is added to the AOF buffer and is added to the rewrite buffer. After the child process completes the AOF rewrite, the main process writes the Write command in the AOF rewrite buffer to the new aof file. AOF background overrides are implemented with new files instead of old files.