Redis persistence and availability

Redis has two types of Persistence: snapshot and aof log files.

Snapshot dB files are binary files with smaller sizes than aof log files. However, the data from the last successful backup time to the down time is lost.

The size of the aof file is a little larger than that of the snapshot, but it is not easy to lose the file.

Currently, redis checks whether there are errors in data files that are supported by snapshots and aof, but the fix is only valid for aof files.

Each snapshot file is a full backup. The principle is to fork a sub-process first, and the parent and child processes share the data domain. Then, the sub-process starts to write data in the shared data domain to a temporary file, and then replace the original backup dB file with atomicity. If a new Write Request comes in during the backup process, a copy page of the data of the current write request is created, and the new write request is updated on the secondary page. This is copy-on-write. In this way, when the file size is very large, the memory will not burst.

The Snapshot file can be triggered by bgsave and save. Of course, you can also configure the number of write requests per second to trigger the backup, if you configure save 100 1000, the backup is triggered when 100 writes are performed in 1000 seconds.

Is the aof file an append? To Aof. Every time redis receives a write request, it is first written to the buffer in the memory, and then refreshed to the disk file. Appendfsync can be used to configure when the disk file is refreshed.

The value of appendfsync can be always, second, or no. Always indicates that when a Write Request comes in, the data is immediately written to the disk. In this way, when a large number of write requests are sent, the disk waits. Second indicates that the operating system writes data in the buffer to the disk per second. No, indicating that the time written to the disk is completely controlled by the operating system (this is critical than the loss, and you do not know how much data will be lost ).

In actual projects, aof files are used for regular sorting. aofrewrite can be configured to add an aof file? How many times or when the minimum size is reached.

 

The above is the persistent content. If the read request is very large and a single redis instance cannot provide high-performance services. In this case, the Master/Slave service can be used. The master redis is responsible for receiving write and read requests, and synchronizing write requests to redis. Read Only from redis.

The principle of data synchronization between the master and slave redis instances is simpler than that of the slave database. The steps are as follows:

1. Send a sync command from redis to the primary master (redis can accept read requests externally or not)

2. The master runs a bgsave command to generate the latest backup file. In this process, it is assumed that a new write request is written to the backlog.

3. When the master generates the latest backup dB file, the master sends it to

4. After receiving the backup file from the redisserver, replace the data in the original redis memory, and then ack the data to the master

5. The master will eventually accept the sync request and start to do bgsave to send all the data backlogs from the redis back to the redis after ack.

6. parse the backlog from redis and put it into the memory.

In practice, it is found that when the Master/Slave node is synchronized, the master redis will inevitably generate a backup file with the latest dB suffix. The aof file is only used when the primary master configures appendonly to yes.

. If the aof and DB Backup files exist at the same time in the primary redis, The aof file synchronization is determined based on whether apsaradb for redis supports Aof.

Note that the synchronized data file is the entire file under the master node each time the Server Load balancer is restarted from the Server Load balancer instance.

In this regard, you can resolve the aof file yourself to achieve incremental synchronization.

Fault recovery:
The primary redis is 10.45.9.114 and port6001.
From redis to 10.45.9.111, port to 6002


When the master fails, select slave as the master. The command is as follows:
Run on slave:
Bgsave and run slaveof no one
Here, slaveof no one will not be serialized to the configuration file. After the restart, the settings in the configuration file will prevail.

Then update the client to point it to the new master, that is, 10.45.9.111 this redis

After the master 10.45.9.114 is restored, use it as the slave of the new master.

Check whether the data file is damaged. aof backup is used here. Therefore, run the following command:
./Redis-check-Aof/home/jbossas/data/redis-data/appendonly. aof
The result is as follows:
Aof analyzed: size = 245, OK _up_to = 245, diff = 0
Aof is valid

If the aof file is corrupted, use the following command to restore it:
./Redis-check-Aof -- fix/home/jbossas/data/redis-data/appendonly. aof

Set the redis configuration file slaveof 10.45.9.114 to slaveof 10.45.9.111 6002.
 
Then restart the instance.

Redis Optimization

Reduce the memory, redis in the following set:
Zsets, lists, hashs
When their element quantity and value size are smaller than a certain limit, ziplist (array) is used to reduce memory storage.
Set the number of shards:
List-max-ziplist-entries 512
List-max-ziplist-value 64

For sets, when the number of elements is smaller than a certain limit, it is also stored as an array, intset
Set the number of shards:
Set-max-intset-entries 512

For the original set, redis stores data in proportion to shard space.

For example, the list set, redis slave uses a linked list to store

Each node stores one data, and this node requires three additional pointers. The three pointers respectively refer to the forward node, the back node, and the current data domain.