[Reprint] Two methods and principles of Redis persistence

Reprinted from http://www.m690.com/archives/371

Redis is an advanced Key-value database. It is similar to memcached, but the data can be persisted and the supported data types are rich. There are strings, lists, collections, and ordered collections. Supports the addition, intersection and complement (difference) of the compute sets on the server side, and supports a variety of sorting functions. So Redis can also be viewed as a data structure server.
All redis data is stored in memory and then periodically asynchronously saved to disk (this is called "semi-persistent mode"), and each data change can be written to a append only file (AOF) (this is called "Full persistence mode").

The first method filesnapshotting: The default redis is to persist data to disk in the form of a snapshot (a binary file, Dump.rdb, this file name can be specified), the format in the configuration file is: Save N m indicates within n seconds , Redis takes snapshots to disk if at least m modifications occur. Of course we can also manually perform save or Bgsave (asynchronous) to take snapshots.

How it works: when Redis needs to be persisted, Redis will fork a child process, the child process writes the data to a temporary RDB file on disk, and when the child process finishes writing the temporary file, the original RDB is replaced, so the advantage is that you can Copy-on-write

Another way to persist is append-only: The Filesnapshotting method is lost when Redis is dead (the amount of data lost depends on the configuration of your save policy), so this is its biggest disadvantage, when the volume of business is large, There is a lot of lost data. The Append-only method can do all the data without loss, but the performance of Redis will be worse. AOF will be able to achieve full-length persistence, only need to open in the configuration file (default is no), appendonly Yes on aof, Redis every time you execute a modified data command, will add it to the AoF file, when the Redis restart, will read the aof file to "replay "To revert to the last moment before Redis was closed.

Log rewriting as the execution of the modified data aof file becomes larger, many of which record the change of a key. So Redis has a more interesting feature: rebuilding the aof file in the background without affecting the client side operation. Executing the bgrewriteaof command at any time will write commands for the shortest sequence in the current memory to disk, which can completely build the current data situation without any extraneous changes (such as state changes, counter changes, etc.) and the size of the aof file. so when using AOF, Redis recommends using bgrewriteaof at the same time.

AoF file Refresh method, there are three, reference configuration parameters Appendfsync :appendfsync always each commit a modification command calls Fsync flush to the aof file, very very slow, but also very safe; appendfsync everysec calls Fsync refresh to aof file every second, soon, but may lose less than one second of data;Appendfsync no relies on OS refresh, Redis does not actively refresh aof, This is the quickest, but the security is poor. The default and recommended refresh per second, so that both speed and security are done.

AOF may be damaged due to system reasons, Redis can no longer load this aof, you may follow the steps below to fix: First make a aof file backup, copy to other places; repair the original aof file, execute:$ redis-check-aof–fix; You can use the Diff–u command to see where files are inconsistent before and after the repair, and restart the Redis service.

LOG rewrite works: The same applies to copy-on-write: First Redis will fork a child process, the child process writes the latest aof to a temporary file, and the parent process increments the most recent execution in memory to write (at this point, the old AoF is still written. Rewrite if the failure is also secure); When the child process finishes rewrite the temporary file, the parent process receives a signal and writes the previous in-memory incremental changes to the end of the temporary file, where Redis renames the old aof file, renames the temporary file, and begins writing to the new aof.

Finally, in case (the machine is broken or the disk is broken), remember to regularly back up the *rdb *.aof files that were generated using filesnapshotting or append-only to the remote machine. I was using crontab every half an hour to SCP once. I did not use Redis's master-slave function, because half-hour backup should be possible, and I think there is a waste of machine if the decision. This eventually depends on the application.

[Reprint] Two methods and principles of Redis persistence