Redis Learning Note Three

First, Redis persistence

Redis is an in-memory database that supports persistence, and Redis needs to frequently synchronize the in-memory data to disk to ensure persistence.

Redis provides different levels of persistence methods:

• Snapshotting (snapshot, default): Ability to snapshot storage of your operations processes at specified intervals
• Append-only file (abbreviated AOF): Records each action that is written to the server and re-executes these commands to restore the original data when the server restarts
• If you only want your data to exist when the server is running, you can also not use any persistence mode
• You can also turn on two persistence methods at the same time, in which case the Redis reboot will first load the AOF originals to restore the original data, since the AoF file is typically saved in a dataset that is more complete than the data set saved by the Rdb file

Second, snapshot mode (snapshotting)

By default, Redis saves a database snapshot in a dump.rdb binary file.

　　

1, Configuration method

You can automatically make snapshot persistence by configuring settings. We can configure Redis to take snapshots automatically if more than M key is modified in n seconds, the following is the default snapshot save configuration:

Save 1  //900 seconds if more than 1 keys are modified, the snapshot is saved save  //300 second content as more than 10 keys are modified, then the snapshot is saved save 60 10000  

2. Working principle

When Redis needs to save the Dump.rdb file, the server performs the following actions:

Redis calls Forks, with both parent and child processes

The child process writes the dataset to a temporary RDB file

When a child process finishes writing to the new Rdb file, Redis replaces the original Rdb file with the new Rdb file and deletes the old Rdb file

The client can also use the Save or Bgsave command to notify Redis to do a snapshot persistence. The save action is to save the snapshot in the main thread, which blocks all client requests because Redis is a main thread to handle all client requests.

Third, append-only files (append-only file AOF)

The snapshot feature is not very durable, and if redis causes downtime for some reason, the server loses the data that was recently written and is still not saved to the snapshot. Starting with version 1.1, Redis adds a completely durable way to persist: aof persistence.

1, Configuration method

AOF is more persistent than snapshot mode. When using the AoF method, Redis appends each received write command to the file (by default, appendonly.aof) through the Write function. When Redis restarts, the contents of the entire database are rebuilt in memory by re-executing the saved write commands in the file.

Because the operating system caches write modifications in the kernel, it may not be written to disk immediately, so the persistence of the AOF mode may also lose some of the modifications. We can tell Redis through the configuration file that we want to force the operating system to write to disk through the Fsync function in three ways (default: Fsync once per second):

appendonly Yes           #启用aof持久化方式 # Appendfsync always   #每次收到写命令就立即强制写入磁盘, the slowest, but guaranteed full persistence, is not recommended to use Appendfsync everysec     #每秒钟强制写入磁盘一次, a good compromise in performance and persistence, recommended # Appendfsync no    #完全依赖os, best performance, no guarantee of durability

Option Analysis:

1, Appendfsync No

When setting Appendfsync to No, Redis does not actively call Fsync to synchronize AOF logs to disk, relying entirely on the operating system

2, Appendfsync everysec

When set to Appendfsync as Everysec, Redis will default to a Fsync call every second, writing data from the buffer to disk.

3, Appendfsync always

When the Appendfsync is set to always, every write operation calls a Fsync, and the data is the safest, and of course, the performance is affected because Fsync is executed every time.

After setting appendonly Yes in Profile redis.windows.conf, open the Redis server and immediately generate a append-only.aof empty file. Then, in the client, enter the following command

　　

View the append-only.aof file again with the following:

　　

2. Log rewriting

Because AOF works by constantly appending commands to the end of a file, the volume of the aof file becomes larger as the write command grows. Redis supports one feature: You can rebuild the AoF file without interrupting the client (rebuild), execute the bgrewriteaof command, and Redis will generate a new aof file that will contain the fewest commands required to rebuild the current dataset.

　　

3. Working principle

The aof rewrite, like the Rdb snapshot, is cleverly leveraging the write-time replication mechanism:

1. Redis Call Fork, now has parent-child two processes
2. A child process writes a command to a temporary file to rebuild the state of a database based on a database snapshot in memory
3. The parent process continues to process the client request, in addition to writing the write command to the original aof file. Cache the received write commands at the same time. This will ensure that if the child process rewrite fails, it will not be problematic.
4. The child process signals the parent process when the child process writes the snapshot content to a temporary file that has been written to the command mode. The parent process then writes the cached write command to the temporary file as well.
5. Now the parent process can replace the old aof file with the temporary file and rename it, and the subsequent write commands are appended to the new AoF file.

4. Write Data Flow

1. The client sends a write request to the server (the data is in the client's memory)
2. The database server receives the data for the write request (the data is in the server's memory)
3. The server calls write this system call and writes the data to disk (the data is in the buffer of the system memory)
4. The operating system transfers data from the buffer to the disk controller (data in the disk buffer)
5. The disk controller writes data to the physical media on the disk (the data actually falls on the disk)

Iv. backing up the Redis database

We can copy the Rdb file when the server is running: Once the Rdb file is created, no modifications are made, and when the server creates a new Rdb file, it saves the contents of the file in a temporary file, and when the temporary file is written, The program will replace the temporary file with the original Rdb file.

• Create a recurring task (cron job) that backs up an RDB file to a folder every hour, and backs up one Rdb file to another folder every day.
• Make sure that the snapshots are backed up with the appropriate date and time information, and use the Find command to delete the expired snapshots each time you execute a recurring task script: For example, you can keep a snapshot per hour for the last 48 hours, and you can keep a daily snapshot for the last month or two.
• At least once a day, back up the RDB to your data center, or at least back up to the physical machine where you run the Redis server.

V. References

1, http://www.redis.cn/topics/persistence.html

Redis Learning Note Three