Redis persistence mechanism

Redis Persistence

Redis data exists in memory, so access performance is good. But there is a problem with the in-memory data, and once the machine restarts, the memory data disappears. To solve this problem, Redis supports persistence. Persistence is to recover data when memory data is lost, rather than to transfer temporarily unused data to the hard disk.

When the Redis storage data reaches the upper memory limit, it will not be saved in the data. In the actual production environment, we'd better ensure that the maximum amount of data is less than half the upper limit of memory. This is the reason behind it.

Cache penetration: A nonexistent value is frequently requested, the cache does not exist, the request database is not present in the database, and each time the database is requested.

Cache avalanche: All caches are invalidated.

Persistence is understood to mean that a copy of the memory data is present on the hard disk so that the memory data can be recovered from the hard disk when it is lost.

Redis supports two ways of persistence, RDB (memory snapshot), and aof (log append). An RDB can also be understood as semi-persistent and is the default persistence for Redis. AOF is better for persistence, but has an impact on performance. Actual production is used in two ways.

VM virtual memory is no longer recommended, severely impacting performance

1:rdb Memory Snapshots

The RDB is called a memory snapshot, and the in-memory data snapshot is periodically written to the hard disk according to the configuration we set.

Principle:

1. Redis calls fork and now has child and parent processes.
2. The parent process continues to process the client request, (without blocking) the child process is responsible for writing the memory contents to the temporary file. Because the OS's write-time replication mechanism (copy on write) will share the same physical page, when the parent process processes the write request, the OS creates a copy of the page to be modified by the parent process, rather than writing the shared page. So the data in the child process's address space is a snapshot of the entire database at fork time. (If there is no write request when the entire subprocess writes data to the hard disk, the parent-child process shares the memory data.) Once a write is done, a copy is copied. If a copy is copied, it will consume double the memory, which is why the production environment is trying to keep the amount of data in half the memory.

3. After the child process writes the snapshot to the temporary file, replaces the original snapshot file with the temporary file, and then the child process exits (the fork is also copied into the internal process, that is, the memory is twice times the original).

The Save and Bgsave commands also trigger an RDB for persistence. The Save command enables the main process to persist, blocking client requests for less. The Bgsave command enables the child process to persist, just as it does above.

RDB persistence loses data from the last backup to the time of the crash, and is good for cold backups. AOF is more appropriate if you want to get better durability. But the performance of aof is worse.

2:aof log append is not turned on by default

AoF will write each of the Redis server writes to the appendonly.aof file, which exists on the hard disk. If memory fails, the Redis server reads the command from this file to recover. The caching mechanism of the operating system itself makes the append of the appendonly.aof not immediately written to the hard disk, and some modification writes are lost when restarting. Redis configuration enables you to force the cache to be written to the hard disk immediately.

The AOF way will make the. aof file more and more large. With configuration, the. aof file can be rewritten on a regular basis. Command bgrewriteaof, you can override the AoF file.

The following is the principle of overriding the. aof file, not the principle of the log append. The log append is appended to the end of the. aof file.

1. Redis Call Fork, now has a parent-child two processes
2. The child process writes to the temporary file a command to rebuild the state of the database (as well as copy-on-write) based on the database snapshot in the in-memory of the fork time
3. The parent process continues to process the client request, in addition to writing the write command to the original aof file. Cache the received write commands at the same time. This will ensure that if the child process rewrite fails, it will not be problematic.
4. When a child process writes the snapshot content to a temporary file, the child process signals the parent process. The parent process then writes the cached write command to the temporary file as well.
5. Now the parent process can replace the old aof file with the temporary file and rename it, and the subsequent write commands are also started to append to the new aof file.

Generally the Redis master from the synchronization, the host does not open any persistent policy, to ensure the best performance. Persistence in a slave-to-machine approach with two strategies

Redis persistence mechanism