AoF of Redis persistence (Append only File) and its summary

1. What is it?

Record each write in the form of log, record all written instructions executed by Redis (not recorded by Read), only append the file but not rewrite the file, Redis start be made early read the file to reconstruct the data, in other words, Redis Restart to complete the recovery of the data according to the contents of the log file to execute the instructions from the front to the back

2.Aof Saves the appendonly.aof file

3. Configuration Location

4.AOP startup, repair, recovery

① Normal Recovery

Start: Set Yes, modify the default AppendOnly no, change to Yes

Copy a aof file with data to the corresponding directory (config get dir)

Recovery: Reboot Redis and Reload

② Exception Recovery

Start: Set Yes, modify the default AppendOnly no, yes

Back up a aof file that was written badly

Repair: redis-check-aof--fix for repair

Recovery: Reboot Redis and Reload

5.rewrite





What ① is.

AOF file Append, file will become larger to avoid this situation, the new rewrite mechanism, when the size of the aof file exceeds the threshold set, Redis will start the AoF file content compression, only the minimum set of data can be recovered, you can use bgrewriteaof

② principle

AOF files continue to grow too large, a new process will be fork to rewrite the file (also to write temporary files last rename), traversing the memory of the new process, each with a set statement. Rewriting the AoF file does not read the old aof file, but instead rewrites the entire memory database content with a new aof file, which is somewhat similar to the snapshot.

③ trigger mechanism

Redis records the AOF size at the time of the last rewrite, which is triggered when the aof file size is one times the size of the last rewrite and the file is greater than 64M

6. Advantages

Sync per modification: Appendfsync always synchronous persistence every time a data change occurs, it is recorded immediately to disk performance is poor but data integrity is better

Sync per Second: Appendfsync everysec asynchronous operation, record per second if there is a second downtime, there is data loss

Out of sync: Appendfsync no never sync

7. Disadvantage

AOF files are much larger than RDB files for data in the same dataset, and recovery is slower than RDB

AoF running efficiency is slower than RDB, the synchronization strategy of each second is better, the synchronization efficiency and RDB are the same

8. Summary

The persistence process of aof

By default, there is no redis_aof.conf configuration file, so we need to replicate a copy, that is, execute the command CP redis.conf redis_aof.conf, and then we vim/myredis/redis_aof.conf Go inside. Change the AppendOnly No, change to appendonly Yes, start the program and set the data, as follows:

root@ubuntu:/usr/local/bin# redis-server/myredis/redis_aof.conf 
root@ubuntu:/usr/local/bin# redis-cli-p 6379
127.0.0.1:6379> Keys *
(empty list or set)
127.0.0.1:6379> Set K1 v1 
OK
127.0.0.1:6379> set K2 v2
OK
127.0.0.1:6379> set K3 v3
OK
127.0.0.1:6379> Flushall
OK
127.0.0.1:6379> SHUTDOWN not
connected> exit

At this point the data is saved to the Appendonly.aof file, and then we open another new terminal window, with the Cat appendonly.aof command as follows:

root@ubuntu:/usr/local/bin# cat appendonly.aof 
*2
$
SELECT
$
0
*3
$
Set
$
K1
$
v1
*3
$
Set
$
K2
$
v2
*3
$
set
$
K3
$
v3
*1
$
Flushall

Then we restart the service again, using the keys * To view the data, found no data, what is the reason.  Because we write the Flushall command in the same time written to the Appendonly.aof file, as shown above, so we entered into the appendonly.aof file, with command dd delete flushall, restart again, you can display data, and then we use VIM Appendonly.aof, go inside and write some data, then restart the service and you'll find

root@ubuntu:/usr/local/bin# redis-server/myredis/redis_aof.conf 
root@ubuntu:/usr/local/bin# redis-cli-p 6379
could not connect to Redis at 127.0.0.1:6379:connection refused

That's because you've just added some characters to the appendonly.aof, so it's an error when you start, which means that you start the AOF configuration file (because Dump.rdb didn't make changes), and it also shows that the two can coexist. So how to fix it.

root@ubuntu:/usr/local/bin# redis-check-aof--fix  appendonly.aof 
0x              6e:expected prefix ' A ', Got: ' * '
aof analyzed:size=151, ok_up_to=110, diff=41
This would shrink the aof from 151 bytes, with a bytes, to bytes
Continue? [y/n]: y
successfully truncated aof

So, we reboot.

root@ubuntu:/usr/local/bin# redis-server/myredis/redis_aof.conf 
root@ubuntu:/usr/local/bin# redis-cli-p 6379
127.0.0.1:6379> Keys *
1) "K3"
2) "K2"
3) "K1"

Summarize:

RDB persistence enables snapshot storage of your data at specified intervals.

AOF record every operation on the server, and when the server restarts, the commands are rerun to restore the original data, the AOF command appends the Redis protocol with each write operation to the end of the file, Redis can also rewrite the aof file in the background, So that the volume of aof files is not too large.

Caching only: If you want your data to exist while the server is running, you may not use any form of persistence.

Turn on two types of persistence at the same time:

① in this case, when the Redis is restarted, the AoF file is loaded as a priority to restore the original data, because in general the AoF file holds a dataset that is more complete than the data set saved by the Rdb file.

②RDB data is not real-time, while using both the server restart will only find aof files, that should not use aof it. The authors recommend not, because RDB is more suitable for backing up the database (aof is constantly changing bad backups), fast restart, and there will be no aof potential bugs, as an in-case approach.