Verifying snapshots and aof of Redis

Brief introduction of Redis persistence:

Redis is a persistent memory database, which means that Redis often synchronizes data in memory to disk to ensure persistence. Redis supports two kinds of persistence, one is snapshotting (snapshot) is also the default, the other is Append-only file (abbreviated AOF) way. The following are described separately

Snapshotting
Snapshots are the default persistence method. In this way, the memory data is written to the binary file as a snapshot, and the default file name is Dump.rdb. You can automatically do snapshot persistence through configuration settings. We can configure Redis to take snapshots automatically if more than M keys are modified in n seconds, following is the default snapshot save configuration

Save 900 1 #900秒内如果超过1个key被修改, a snapshot save is initiated
Save #300秒内容如超过10个key被修改, the snapshot is saved
Save 60 10000

The detailed snapshot save process is described below

1.redis calls Fork, now has child processes and parent processes.

2. The parent process continues to process the client request, which is responsible for writing the memory contents to the temporary file. Because the OS's write-time replication mechanism (copy on write) parent-child processes share the same physical page, the OS creates a copy of the page that the parent process is modifying, rather than a shared page, when the parent process processes the write request. So the data in the address space of the child process is a snapshot of the entire database at fork time.

3. When the child process writes the snapshot to the temporary file, replace the original snapshot file with a temporary file, and the child process exits.

The client can also use the Save or Bgsave command to notify Redis to do a snapshot persistence. The save operation saves the snapshot in the main thread, which blocks all client requests because Redis is a main thread to handle all client requests. So it is not recommended to use. Another point to note is that each snapshot persistence is a complete write of the memory data to the disk, not an incremental synchronization of dirty data only. If the amount of data is large, and more write operations, will inevitably cause a lot of disk IO operations, may seriously affect performance.

In addition, because the snapshot is done at a certain interval, if Redis accidentally down, all changes after the last snapshot are lost. If the application requirements can not be lost any changes, the use of aof persistence. Described below

Append-only file

AOF is more persistent than snapshot, because when you use the AoF persistence method, Redis appends each received write command to the file (by default, appendonly.aof) through the Write function. When Redis restarts, the contents of the entire database are rebuilt in memory by executing the write commands saved in the file. Of course, because the OS caches write modifications in the kernel, it may not be written to disk immediately. In this way the persistence of aof is also likely to lose some of the changes. But we can tell Redis through the configuration file that we want to force the OS to write to disk via the Fsync function. There are three ways to do this (the default is: Fsync once per second)

AppendOnly Yes//enable AOF persistence mode
# Appendfsync always//force write to disk every time you receive a write command, slowest, but guaranteed to be completely persistent, deprecated
Appendfsync everysec//second forced write disk once, a good compromise in performance and persistence, recommended
# Appendfsync no//completely dependent on OS, best performance, no guarantee of persistence

The AOF approach also brings up another problem. The persisted file will become larger and bigger. For example, we call the INCR Test command 100 times, the file must save all 100 commands, in fact, 99 are redundant. Because you want to restore the state of the database in fact, a set test 100 is enough to save the file. To compress the persisted files of the aof. Redis provides the bgrewriteaof command. Receive this command Redis will save the data in memory to a temporary file in a way that is similar to the snapshot, and finally replace the original file. The specific process is as follows

1. Redis calls Fork, now has a parent-child two process
2. The child process writes a command to rebuild the database state to a temporary file based on a database snapshot in memory
3. The parent process continues to process the client request, in addition to writing the write command to the original aof file. At the same time, the received write commands are cached. This ensures that if the child process fails to rewrite, it will not be a problem.
4. When the child process writes the snapshot contents to a temporary file in a command way, the child processes signal the parent process. The parent process then writes the cached write command to the temporary file.
5. Now the parent process can replace the old aof file with the temporary file and rename it, and the write commands that are received later are also appended to the new AoF file.

Note that rewriting the AoF file does not read the old aof file, but instead rewrites the entire memory database content with a command to rewrite a new aof file, which is somewhat similar to the snapshot.

Http://www.cnblogs.com/xuxm2007/archive/2011/11/28/2265894.html

Redis boot mount:

AOF priority over Rdb
RDB performance is superior to aof because there is no repetition inside
Redis load data into memory at once, preheat at a one-time

First, verify the Redis snapshot function

1. Modify Redis configuration file
--The following redis.conf configuration file parameter values are default and can be modified as needed
[Root@rac1 redis-2.6.8]# vi/etc/redis_master.conf
# snapshot Save rules, such as
Save 900 1 #900秒内如果超过1个key被修改, a snapshot save is initiated
Save #300秒内容如超过10个key被修改, the snapshot is saved
Save 60 10000

# Maximum number of databases used
Databases 16

# Name of the snapshot file
Dbfilename Dump.rdb

# snapshots stored in the directory
dir/opt/redis-2.6.8

# Whether the data file is to be compressed. (set to No for easy viewing of data file contents)
Rdbcompression No

# Redis The password for the operation
# Requirepass Foobared

# Append only File feature is turned off (note: To better test the snapshot functionality, this feature is best closed)
AppendOnly No

# Daemonize: Specifies whether Redis runs in the background

Daemonize Yes

2. Start Redis
[Root@rac1 ~]# redis-server/etc/redis_master.conf > Redis_master.log 2>&1 &
[1] 5009

3. Command Test Snapshot Save
[Root@rac1 redis-2.6.8]# Redis-cli
Redis 127.0.0.1:6379> Keys *
(empty list or set)
Redis 127.0.0.1:6379> Set AAA 1
Ok
Redis 127.0.0.1:6379> Get AAA
"1"
Redis 127.0.0.1:6379> set BBB 2
Ok
Redis 127.0.0.1:6379> Set CCC 3
Ok
Redis 127.0.0.1:6379> Save
Ok
Redis 127.0.0.1:6379> Set DDD 4
Ok
Redis 127.0.0.1:6379> Keys *
1) "AAA"
2) "BBB"
3) "CCC"
4) "DDD"
Redis 127.0.0.1:6379> exit

--kill off the Redis process
[Root@rac1 redis-2.6.8]# kill-9 5009
[Root@rac1 redis-2.6.8]#
[1]+ has killed redis-server/etc/redis_master.conf > Redis_master.log 2>&1

--View Redis_master.log Log
[30640] Mar 12:06:07.037 * DB saved on disk #save命令后数据保存到磁盘
[30640] Mar 12:06:07.043 * rdb:0 MB of memory used by Copy-on-write
[30637] Mar 12:06:07.063 * Background saving terminated with success
[30637] Mar 12:06:07.069 * Synchronization with Slave succeeded
[30637] Mar 12:07:23.806 * DB saved on disk

4. View Data File contents
[Root@rac1 redis-2.6.8]# LL
-rw-rw-r--1 root root 14692 01-11 00:15 00-releasenotes
-rw-rw-r--1 root 01-11 00:15 BUGS
-rw-rw-r--1 root root 1440 01-11 00:15 contributing
-rw-rw-r--1 root root 1487 01-11 00:15 copying
Drwxrwxr-x 6 root root 4096 03-12 16:07 deps
-rw-r--r--1 root 03-18 11:59 Dump.rdb
-rw-rw-r--1 root 01-11 00:15 INSTALL
-rw-rw-r--1 root root 151 01-11 00:15 Makefile
-rw-rw-r--1 root root 4038 01-11 00:15 Manifesto
-rw-rw-r--1 root root 4059 01-11 00:15 README
-rw-rw-r--1 root root 23259 03-18 11:48 redis.conf
-rwxrwxr-x 1 root root 160 01-11 00:15 runtest
-rw-rw-r--1 root root 5754 01-11 00:15 sentinel.conf
Drwxrwxr-x 2 root root 4096 03-12 16:11 src
Drwxrwxr-x 8 root root 4096 01-11 00:15 tests
Drwxrwxr-x 2 root root 4096 01-11 00:15 utils

--Restart Redis again, load the snapshot, and discover that the unsaved ddd is missing
[Root@rac1 redis-2.6.8]# redis-server/etc/redis_master.conf > Redis_master.log 2>&1 &
[1] 7113
[Root@rac1 redis-2.6.8]# Redis-cli
Redis 127.0.0.1:6379> Keys *
1) "AAA"
2) "BBB"
3) "CCC"

[Root@rac1 redis-2.6.8]# Cat Dump.rdb
Redis0006taaaàbbbàcccà?l
? O￠w? [Root@rac1 redis-2.6.8]#

Second, verify the AOF function of Redis
As with the snapshot function, implementing the AOF function requires only modifying configuration parameters

1. Modify REDIS.CONF configuration file
--Main modification appendonly
[Root@rac1 redis-2.6.8]# vi/etc/redis_master.conf
AppendOnly Yes

2. Restart Redis Service
[Root@rac1 redis-2.6.8]# redis-cli shutdown
[16739] 13:49:18.232 # User requested shutdown ...
[16739] 13:49:18.234 * Saving the final RDB snapshot before.
[16739] Mar 13:49:18.245 * DB saved on disk
[16739] 13:49:18.245 # Redis is now ready to exit, Bye bye ...
[1]+ Done redis-server/etc/redis_master.conf

[Root@rac1 redis-2.6.8]# redis-server/etc/redis_master.conf &

[Root@rac1 redis-2.6.8]# LL
-rw-r--r--1 root 0 03-18 13:54 appendonly.aof

3. Command test
[Root@rac1 redis-2.6.8]# Redis-cli
Redis 127.0.0.1:6379> Keys *
(empty list or set)
Redis 127.0.0.1:6379> Set eee 1
Ok
Redis 127.0.0.1:6379> Set XXX 2
Ok
Redis 127.0.0.1:6379> Get Eee
"1"
Redis 127.0.0.1:6379> Get XXX
"2"
Redis 127.0.0.1:6379> Keys *
1) "XXX"
2) "Eee"

--View Log files
[Root@rac1 redis-2.6.8]# Cat Appendonly.aof
*2
$
SELECT
$
0
*3
$
Set
$
Eee
$
1
*3
$
Set
$
Xxx
$
2