Redis Data Backup and recovery

Redis Data Backup and recoveryPersistence of

First, RDB mode (default)

The RDB approach is persisted through snapshots (snapshotting), and Redis automatically snaps all of the data in memory and stores it on the hard disk when certain conditions are met. The conditions for a snapshot can be customized by the user in the configuration file, consisting of two parameters: the number of times and changes in the key. A snapshot is taken when the number of keys changed within a specified amount of time is greater than the specified number. The RDB is the default persistence for Redis, with 3 conditions already provisioned in the configuration file:

Save 900 1 # 900 seconds with at least 1 keys changed then take snapshot save 300 10 # 300 seconds with at least 10 keys changed then take snapshot save 60 10000 # 60 seconds with at least 10,000 keys changed

There can be more than one condition, the relationship between the conditions is "or", and a snapshot is taken whenever one of the conditions is met. If you want to disable automatic snapshots, simply delete all of the save parameters.

Redis defaults to storing snapshot files in the Dump.rdb file of the current directory (configurable get dir to view), and you can specify the storage path and file name of the snapshot file by configuring Dir and dbfilename two parameters, respectively.

The process of implementing a snapshot with Redis

1. Redis uses the fork function to copy a copy of the current process (the parent process) (the child process);

2. The parent process continues to receive and process commands from the client, and the child process begins to write the data in memory to a temporary file on the hard disk;

3. When the child process finishes writing all the data, it replaces the old Rdb file with the temporary file, and the snapshot operation is completed.

The operating System (Unix-like operating system) uses the write-time replication (Copy-on-write) policy when the fork is executed, that is, when the fork function occurs, the parent-child process shares the same memory data, and when the parent process changes one of the pieces of data (such as executing a write command), The operating system copies the piece of data to ensure that the child process's data is not affected, so the new Rdb file stores the memory data that executes the fork moment.

Redis does not modify the Rdb file during a snapshot, and the old file is replaced with the new one at the end of the snapshot, which means that the Rdb file is complete at any time. This allows us to backup the Redis database by regularly backing up the RDB files. An RDB file is a binary format that is compressed (you can configure the Rdbcompression parameter to disable compression to conserve CPU), so the space used is less than the size of the in-memory data and is more conducive to transmission.

In addition to automatic snapshots, you can manually send the Save or bgsave command to have Redis perform a snapshot, the difference between the two commands is that it is a snapshot operation by the main process, which blocks other requests, which take a snapshot operation through the fork subprocess. After Redis boots, the Rdb snapshot file is read and the data is loaded into memory from the hard disk. This time is different depending on the size of the data and the structure and server performance. It typically takes 20-30 seconds to load a snapshot file with 10 million string type keys and a size of 1GB into memory. By using the Rdb method to persist, all data changed after the last snapshot is lost once the Redis exits unexpectedly. This requires the developer to control the possible loss of data in an acceptable range by combining the set of automatic snapshot conditions in a specific application. If the data is important enough to withstand any loss, consider using the AoF method to persist.

Second, the AOF way

By default, Redis does not have a persistence of aof (append only file), which can be turned on by the appendonly parameter in redis.conf:

AppendOnly Yes

At startup, Redis executes the commands in the AoF file one by one to load the data from the hard disk into memory, which is slower than the RDB

When AOF is turned on for each execution of a command that changes data in Redis, Redis writes the command to the AoF file on the hard disk. The aof file is saved in the same location as the Rdb file, which is set by the Dir parameter, and the default file name is Appendonly.aof, which can be modified by the Appendfilename parameter:

Appendfilename appendonly.aof

To configure the conditions for Redis to automatically override aof files

Auto-aof-rewrite-percentage 100 # When the current aof file size exceeds the AoF file size of the last rewrite, it will be rewritten again if not previously rewritten, The aof file size at startup is based on the Auto-aof-rewrite-min-size 64MB # allowed to override the minimum aof file size

A mechanism that requires the system to flush the hard disk cache after the configuration is written to the AoF file

# Appendfsync always # performs synchronization every time a write is performed, the safest and slowest Appendfsync Everysec # performs a synchronous operation once per second # Appendfsync No # does not actively synchronize, but is completely left to the operation System to do (i.e. every 30 seconds), the fastest and least secure

Redis allows both aof and RDB to be enabled, ensuring data security and making backup operations easy. After you restart Redis, Redis uses the aof file to recover the data because the AoF method may lose less data for persistence

Copy

With the persistence feature, Redis guarantees that there will be no loss (or small loss) of data even if the server is restarted. However, because the data is stored on a single server, if the server's hard disk fails, it can also cause data loss. To avoid a single point of failure, we want to replicate multiple copies of the database to be deployed on different servers, even if one server fails and other servers continue to serve. This requires that when the database on one server is updated to automatically synchronize the updated data to other servers, Redis provides the process of synchronizing the replication (replication) functionality automatically.

Configuration Method

• The configuration file is added from the database configuration file slaveof master-ip master-port , the primary database does not need to be configured

• When starting Redis-server with command-line arguments, use the command-line arguments--slaveof master-ip master port

Redis-server--port 6380--slaveof 127.0.0.1 6379

by commandSLAVEOF master-ip master-port

redis>slaveof 127.0.0.1 6379

• SLAVEOF NO ONECan be the current database stop receiving synchronization from other databases, and go to the primary database

advantages and Application Scenarios

1. Read-write separation enables read-write separation to increase server load capacity through replication. In a common scenario, the frequency of reading is greater than the write, when a single-machine redis cannot cope with a large number of Read requests (especially the more resource-intensive requests, such as the sort command, etc.) can be established through the replication function from the database, the primary database only write operations, and from the database is responsible for read operations.

2. Persistent persistence from a database is often relatively time-consuming, in order to improve performance, you can create one (or several) from the database through replication, and enable persistence from the database while disabling persistence in the primary database. Restarting the base database when it crashes from the database automatically synchronizes the data so there is no need to worry about data loss. When the primary database crashes, you need to use the slaveof NO one command from the database to promote the service from the database to the primary database, and then use the slaveof command to set it to the new primary database from the database after the original primary database is started, and then synchronize the data back.

Redis Data Backup and recovery