Redis Data Backup and recovery

Persistence of

RDB Mode (default)

The RDB approach is persisted through snapshots (snapshotting), and Redis automatically snaps all of the data in memory and stores it on the hard disk when certain conditions are met. The conditions for a snapshot can be customized by the user in the configuration file, consisting of two parameters: the number of times and changes in the key. A snapshot is taken when the number of keys changed within a specified amount of time is greater than the specified number. The RDB is the default persistence for Redis, with 3 conditions already provisioned in the configuration file:

Save 1    # 900 seconds with at least 1 keys changed to take snapshot
save   # 300 seconds There are at least 10 keys being changed then take a snapshot
save 60 10000 # Snapshot of at least 10,000 keys changed in 60 seconds

There can be more than one condition, the relationship between the conditions is "or", and a snapshot is taken whenever one of the conditions is met. If you want to disable automatic snapshots, simply delete all of the save parameters.

Redis defaults to storing snapshot files in the Dump.rdb file of the current directory (configurable get dir to view), and you can specify the storage path and file name of the snapshot file by configuring Dir and dbfilename two parameters, respectively.

Redis implementation of snapshots Redis uses the fork function to copy a copy of the current process (the parent process), the parent process continues to receive and process commands from the client, and the child process begins to write the data in memory to a temporary file on the hard disk; When the child process finishes writing all the data, it replaces the old Rdb file with the temporary file, and the snapshot operation is completed.

The operating System (Unix-like operating system) uses the write-time replication (Copy-on-write) policy when the fork is executed, that is, when the fork function occurs, the parent-child process shares the same memory data, and when the parent process changes one of the pieces of data (such as executing a write command), The operating system copies the piece of data to ensure that the child process's data is not affected, so the new Rdb file stores the memory data that executes the fork moment.

Redis does not modify the Rdb file during a snapshot, and the old file is replaced with the new one at the end of the snapshot, which means that the Rdb file is complete at any time. This allows us to backup the Redis database by regularly backing up the RDB files. An RDB file is a binary format that is compressed (you can configure the Rdbcompression parameter to disable compression to conserve CPU), so the space used is less than the size of the in-memory data and is more conducive to transmission.

In addition to automatic snapshots, you can manually send the Save or bgsave command to have Redis perform a snapshot, the difference between the two commands is that it is a snapshot operation by the main process, which blocks other requests, which take a snapshot operation through the fork subprocess. After Redis boots, the Rdb snapshot file is read and the data is loaded into memory from the hard disk. This time is different depending on the size of the data and the structure and server performance. It typically takes 20-30 seconds to load a snapshot file with 10 million string type keys and a size of 1GB into memory. By using the Rdb method to persist, all data changed after the last snapshot is lost once the Redis exits unexpectedly. This requires the developer to control the possible loss of data in an acceptable range by combining the set of automatic snapshot conditions in a specific application. If the data is important enough to withstand any loss, consider using the AoF method to persist.

AoF Way

By default, Redis does not have a persistence of aof (append only file), which can be turned on by the appendonly parameter in redis.conf:

AppendOnly Yes

At startup, Redis executes the commands in the AoF file one by one to load the data from the hard disk into memory, which is slower than the RDB

When AOF is turned on for each execution of a command that changes data in Redis, Redis writes the command to the AoF file on the hard disk. The aof file is saved in the same location as the Rdb file, which is set by the Dir parameter, and the default file name is Appendonly.aof, which can be modified by the Appendfilename parameter:

Appendfilename appendonly.aof

To configure the conditions for Redis to automatically override aof files

Auto-aof-rewrite-percentage  # When the current aof file size exceeds the AoF file size of the last rewrite, it is rewritten again, based
on the aof file size at startup if it was not previously rewritten Auto-aof-rewrite-min-size 64MB   # Allows overriding of the minimum aof file size

A mechanism that requires the system to flush the hard disk cache after the configuration is written to the AoF file

# Appendfsync   always # performs synchronization every time a write is performed, the safest and slowest
Appendfsync everysec   # Performs a synchronous operation once per second
# Appendfsync no       # Do not take the initiative to synchronize, but completely to the operating system (that is, every 30 seconds), the fastest and most insecure

Redis allows both aof and RDB to be enabled, ensuring data security and making backup operations easy. After you restart Redis, Redis uses the aof file to recover data because the persistence of the AoF method may result in less data being copied

With the persistence feature, Redis guarantees that there will be no loss (or small loss) of data even if the server is restarted. However, because the data is stored on a single server, if the server's hard disk fails, it can also cause data loss. To avoid a single point of failure, we want to replicate multiple copies of the database to be deployed on different servers, even if one server fails and other servers continue to serve. This requires that when the database on one server is updated to automatically synchronize the updated data to other servers, Redis provides the process of synchronizing the replication (replication) functionality automatically. Configuration Method

Slaveof Master-ip Master-port is added to the configuration file of the database from the configuration file, the primary database does not need to be configured

Use command-line arguments--slaveof MASTER-IP Master Port when starting redis-server with command-line arguments

Redis-server--port 6380--slaveof 127.0.0.1 6379

by command slaveof Master-ip Master-port

redis>slaveof 127.0.0.1 6379

Slaveof NO One can be the current database stop receiving synchronization of other databases, turn into the main database advantages and application Scenarios

Read-write separation enables read-write separation to increase server load capacity through replication. In a common scenario, the frequency of reading is greater than the write, when a single-machine redis cannot cope with a large number of Read requests (especially the more resource-intensive requests, such as the sort command, etc.) can be established through the replication function from the database, the primary database only write operations, and from the database is responsible for read operations.

Persistent persistence from a database is often relatively time-consuming, in order to improve performance, you can create one (or several) from the database through replication, and enable persistence from the database while disabling persistence in the primary database. Restarting the base database when it crashes from the database automatically synchronizes the data so there is no need to worry about data loss. When the primary database crashes, you need to use the slaveof NO one command from the database to promote the service from the database to the primary database, and then use the slaveof command to set it to the new primary database from the database after the original primary database is started, and then synchronize the data back.