Redis Persistent AOF RDB

The AOF persistence strategy for Redis is to record every command sent to the Redis server and save it to the AoF file on the hard disk, like a log file, and a single command.

AOF settings

The location of the AoF file is the same as the location of the Rdb file, which is set by the Dir parameter, and the default file name is Appendonly.aof, which can be modified by appendfilename parameters.

AOF Test

When a client sends some REDIS commands to the server, Redis logs the executed command to the AoF file as follows:

650) this.width=650; "src=" http://img.blog.csdn.net/20160630175152955 "alt=" here write a picture describing "title=" "style=" border:none; "/ >

When the Redis server restarts, the aof file will be executed to achieve the purpose of data recovery.

aof file Rewrite

Why rewrite? Overrides can remove intermediate execution of data and retain the final data command directly.

Give a chestnut: for example, a Redis client performs a series of commands on key

Set count 1//initial value is 1
INCR count//plus 1
INCR count//plus 1
DECR count//minus 1

This time the result is

Get Count
"2"

If the aof rewrite is not performed, Redis executes each command in the AoF file and ends up with a count of 2 when the AoF file is restored.

After aof rewriting, it is equivalent to omitting the intermediate calculation process. Setting the computed results directly into Redis is equivalent to executing only one command set, Count 2.

You can use the bgrewriteaof command to override the AoF file.

overriding policy

Override the parameter settings for the policy:

Auto-aof-rewrite-percentage 100

When the current aof file size exceeds the size of the AoF file at the time of the last rewrite, it is rewritten again, based on the size of the AoF file at startup if it was not previously rewritten.

Auto-aof-rewrite-min-size 64MB

The minimum aof file size that is allowed to be overridden is limited, usually when the aof file is small, even if some of the redundant commands are negligible.

Redis has excellent performance because it stores all of the data in memory, as well as memcached, but why is redis able to stand out, largely because Redis has a great persistence mechanism that ensures that data is not lost after the server restarts. Let's look at how Redis is persistent.

Redis supports two ways of persistence, one for RDB and one for aof. Either method can be used alone, or mixed.

Introduction to the Rdb method

The Rdb method is done through a snapshot, and Redis automatically snaps all the data in memory to the hard disk when certain conditions are met. Just like taking pictures, save everything for the moment. The conditions for the snapshot are specified in the configuration file. There are two main parameters: the number of times and changes in the key value, that is, when the number of keys changed within a specified time is greater than the value of the execution, the snapshot takes place. An RDB is the default persistence mode for Redis.

RDB Mode configuration

Locate Redis configuration file: redis.conf

1) Set the trigger condition:

650) this.width=650; "src=" http://img.blog.csdn.net/20160630161447993 "alt=" here write a picture describing "title=" "style=" border:none; "/ >

650) this.width=650; "src=" http://img.blog.csdn.net/20160630161530791 "alt=" here write a picture describing "title=" "style=" border:none; "/ >

2) Set the Rdb file path

The default RDB file storage path is the current directory and the file name is: Dump.rdb. You can modify the path and file name in the configuration file, respectively dir and Dbfilename

650) this.width=650; "src=" http://img.blog.csdn.net/20160630161844807 "alt=" here write a picture describing "title=" "style=" border:none; "/ >

After Redis starts, it reads the RDB snapshot file, loads the data from the hard disk into memory, and generally takes approximately 20-30 minutes to load the snapshot file into memory in 1GB.

How the RDB makes a snapshot

Snapshot process for RDB:

1) Redis uses the fork function to copy a copy of the current process (parent process);

2) The parent process continues to accept and process commands from the client, and the child process begins to write the in-memory data to the temporary file on the hard disk;

3) When the child process has finished writing all the data, it replaces the old Rdb file with the temporary file.

Manual snapshot:

If you do not trigger an automatic snapshot, you can take a manual snapshot of Redis, both save and Bgsave can perform a manual snapshot, and the difference between the two commands is that the primary process performs a snapshot operation that blocks other requests, while the latter is a snapshot operation through the fork subprocess.

Attention:

Because Redis uses fork to replicate a copy of the current process, the child process will occupy the same memory resources as the main process, such as the main process 8G memory, then the backup must be guaranteed to have 16G of memory, otherwise it will enable virtual memory, performance is very poor.

Compression of RDB files

When an RDB file is too large, it can be compressed, Redis turns on compression by default, and of course you can disable compression by configuring the Rdbcompression parameter.

650) this.width=650; "src=" http://img.blog.csdn.net/20160630173614310 "alt=" here write a picture describing "title=" "style=" border:none; "/ >

Advantages and disadvantages of compression and non-compression:

Compression:

Advantage: Reduce disk storage space
Cons: Consuming CPU resources

Do not compress:

Advantage: Do not consume CPU resources
Cons: More disk space usage

RDB Benefits
An RDB is a compact file that represents the Redis data for an instant point. The RDB file is suitable for backup. For example, you might want to archive an RDB file for the last 24 hours per hour, saving an RDB snapshot for nearly 30 days every day. This allows you to easily recover different versions of the data set for disaster tolerance.
The RDB is ideal for disaster recovery, as a compact single file that can be transferred to a remote data center or Amazon S3 (possibly encrypted).
The RDB maximizes the performance of Redis because the only thing that needs to be done when the Redis parent process is persisted is to start (fork) a subprocess and complete all remaining work by the child process. The parent process instance does not need to perform operations such as disk IO.
The RDB is faster than AOF when restarting an instance of a large data set.
RDB Disadvantages
When you need to minimize data loss when Redis stops working (for example, a power outage), the RDB may not be very good. You can configure different save points. However, you typically create an RDB snapshot every 5 minutes or more, so once Redis stops working for any reason that doesn't shut down properly, you'll have to prepare for data loss in the last few minutes.
An RDB needs to call the fork () child process frequently to persist to disk. If the data set is large, the fork () is time consuming, and the result is that Redis stops the service client for milliseconds or even seconds when the dataset is very large and the CPU performance is not strong enough. AOF also needs fork (), but you can adjust how often you rewrite the log without compromising (trade-off) persistence (durability).
AOF Advantages
Using AOF Redis will be more durable (durable): You can have many different Fsync strategies: no Fsync, Fsync per second, fsync on each request. With the default Fsync policy per second, write performance is still good (Fsync is done by a background thread, the main thread continues to work hard to write requests), even if you only lose one second of write data.
The AOF log is an append file, so there is no need to locate and there is no damage when power is lost. Even if for some reason the end of the file is a half-written command (disk full or other reason), the Redis-check-aof tool can be easily repaired.
The AOF file contains one operation after another, stored in a format that is easy to understand and parse. You can also easily export a AOF file. For example, even if you accidentally use the Flushall command to clear everything, if you do not perform the rewrite at this time, you can still save your dataset, you just have to stop the server, delete the last command, and then restart Redis.
AOF Disadvantages
For the same data set, the AOF file is usually larger than the equivalent RDB file.
AOF may be slower than an RDB, depending on the exact fsync strategy. Usually the Fsync is set to one time per second and the performance is still high, and if the Fsync is turned off, it is as fast as an RDB even under high load. However, an RDB can provide a good maximum delay guarantee even in the case of a large write load.

How RDB and AoF choose
In general, you should use both of these persistence methods to achieve the same level of data security as PostgreSQL provides.
If you are concerned about your data, but you can still accept a few minutes of data loss during a disaster, you can only use the RDB alone.
There are many users who use AOF alone, but we do not encourage this, because it is very convenient to take an RDB snapshot very easily for database backups, faster to boot, and to avoid bugs in the AOF engine.

How to choose? Then you need to look at the requirements and look at the server resources.

This article is from the "Dream to Reality" blog, please be sure to keep this source http://lookingdream.blog.51cto.com/5177800/1941715

Redis Persistent AOF RDB