Detailed analysis on the AOF principle of Redis data persistence Mechanism

The source code referenced in this article is all from Redis2.8.2. The configuration file of Redis is redis. conf. For the parsing code of the configuration file, see config. c Original article reprinted, please note that this article is from blog. csdn. netacceptedxukaiarticledetails18135219AOF Implementation Mechanism AOF is AppendOnlyFile. Implementation Mechanism: R

The source code referenced in this article is all from Redis2.8.2. The configuration file of Redis is redis. conf. For the parsing code of the configuration file, see config. c Original article reprinted please note that this article from the http://blog.csdn.net/acceptedxukai/article/details/18135219 AOF Implementation Mechanism AOF that Append Only File, implementation mechanism: R

The source code referenced in this article is all from Redis2.8.2.

The configuration file of Redis is redis. conf. For the parsing code of the configuration file, see config. c.

Original article reprinted please note, this article from http://blog.csdn.net/acceptedxukai/article/details/18135219

AOF Implementation Mechanism

AOF is Append Only File. Implementation Mechanism: Redis takes a snapshot of the database, traverses all databases, and restores the data in the database to the Protocol format string of the Command sent from the client, then Redis creates a temporary file to save the snapshot data. After the snapshot program ends, the temporary file name is changed to the normal aof file name, and the original file is automatically discarded, because some new commands may modify the data in the database during the snapshot process, you need to append the new data to the aof file after the snapshot program ends, subsequent commands from the client will be constantly written to the disk based on different security levels. In this way, real-time persistence is supported, but data may be lost in a short period of time, which is acceptable to general systems.

Description of AOF parameter configuration in the configuration file redis. conf appendonly

Meaning: whether the AOF persistence mechanism is enabled for Redis Server

The parsing code of the config. c function loadServerConfigFromString is

if (!strcasecmp(argv[0],"appendonly") && argc == 2) {int yes;if ((yes = yesnotoi(argv[1])) == -1) {err = "argument must be 'yes' or 'no'"; goto loaderr;}server.aof_state = yes ? REDIS_AOF_ON : REDIS_AOF_OFF;}

Appendfsync

Description: Redis refresh the data in the OS data buffer to the disk.

# Appendfsync always fsync as long as there is newly added data
Appendfsync everysec supports fsync latency
# No fsync is required for appendfsync
Parsing Code in config. c

If (! Strcasecmp (argv [0], "appendfsync") & argc = 2) {// AOF fsync policy if (! Strcasecmp (argv [1], "no") {server. aof_fsync = AOF_FSYNC_NO;} else if (! Strcasecmp (argv [1], "always") {server. aof_fsync = AOF_FSYNC_ALWAYS;} else if (! Strcasecmp (argv [1], "everysec") {server. aof_fsync = AOF_FSYNC_EVERYSEC;} else {err = "argument must be 'No', 'Always 'or 'everysec'"; goto loaderr ;}}

No-appendfsync-on-rewrite

Meaning: When the rewrite AOF sub-process or RDB sub-process is executing, does the Server support fsync, that is, whether to refresh the data to the hard disk after the newly modified data is written to the AOF file?

Parsing Code in config. c

if (!strcasecmp(argv[0],"no-appendfsync-on-rewrite") && argc == 2) {if ((server.aof_no_fsync_on_rewrite= yesnotoi(argv[1])) == -1) {err = "argument must be 'yes' or 'no'"; goto loaderr;}}

Server. aof_no_fsync_on_rewrite controls the implementation code of this mechanism in the flushAppendOnlyFile function of aof. c. The specific code is

// If fsync is not supported, or the aof rdb sub-process is running, return directly. // but the data has been written to the aof file, but is not refreshed to the hard disk if (server. aof_no_fsync_on_rewrite & (server. aof_child_pid! =-1 | server. rdb_child_pid! =-1) return;

Appendfilename

Description: AOF file name

Auto-aof-rewrite-percentage 100

Auto-aof-rewrite-min-size 64 mb

These two configurations are not explained for the time being. See the AOF parameter comments in the redisServer struct below.

AOF parameters in the redisServer Data Structure

Struct redisServer {............ /* AOF persistence */int aof_state;/* REDIS_AOF _ (ON | OFF | WAIT_REWRITE) */int aof_fsync;/* Kind of fsync () policy */char * aof_filename; /* Name of the AOF file */int aof_no_fsync_on_rewrite;/* Don't fsync if a rewrite is in prog. */int aof_rewrite_perc;/* Rewrite AOF if % growth is> M and... */off_t aof_rewrite_min_size;/* the AOF file is at least N bytes. */aof file minimum byte off_t aof_rewrite _ Base_size;/* AOF size on latest startup or rewrite. */The size of the last aof file off_t aof_current_size;/* AOF current size. */The current aof file size int aof_rewrite_scheduled;/* Rewrite once BGSAVE terminates. */pid_t aof_child_pid;/* PID if rewriting process */pid list of sub-Processes * aof_rewrite_buf_blocks;/* Hold changes during an AOF rewrite. */sds aof_buf;/* AOF buffer, written before entering the event loop */int Of_fd;/* File descriptor of currently selected AOF file */aof File descriptor int aof_selected_db;/* Currently selected DB in AOF */time_t aof_flush_postponed_start; /* UNIX time of postponed AOF flush */time_t aof_last_fsync;/* UNIX time of last fsync () */time_t aof_rewrite_time_last;/* Time used by last AOF rewrite run. */time_t aof_rewrite_time_start;/* Current AOF rewrite start time. */int aof _ Lastbgrewrite_status;/* REDIS_ OK or REDIS_ERR */unsigned long aof_delayed_fsync;/* delayed AOF fsync () counter */int handle;/* fsync incrementally while rewriting? */............}

The string aof_buf and the linked list aof_rewrite_buf_blocks are important parameters for appending the AOF file and rewrite AOF file post-processing differences.

InitServerConfig function initialization and comment on AOF Parameters

Server. aof_state = REDIS_AOF_OFF; // whether the server is enabled for the AOF file. aof_fsync = REDIS_DEFAULT_AOF_FSYNC; // fsync policy. The default value is fsyncserver per second. aof_no_fsync_on_rewrite = REDIS_DEFAULT_AOF_NO_FSYNC_ON_REWRITE; // whether fsync is supported when the aof rdb sub-process is running. After writing the AOF file, refresh the data to the hard disk server. aof_rewrite_perc = REDIS_AOF_REWRITE_PERC; // auto rewrite increment value server. aof_rewrite_min_size = REDIS_AOF_REWRITE_MIN_SIZE; // the minimum number of bytes of the AOF file server. aof_rewrite_base_size = 0; // auto rewrite calculates the base of the aof file increment, which is equal to the number of bytes of the last aof file server. aof_rewrite_scheduled = 0; // rewrite task plan. When the client sends the bgrewriteaof command, if the current rewrite sub-process is executing, the bgrewriteaof requested by the client is changed to a scheduled task, run rewriteserver after AOF sub-process ends. aof_last_fsync = time (NULL); // The Last fsync data to the hard disk time server. aof_rewrite_time_last =-1; // rewrite duration server. aof_rewrite_time_start =-1; // server at which rewrite starts. aof_lastbgrewrite_status = REDIS_ OK; // status server after rewrite. aof_delayed_fsync = 0; // number of times that fsync is delayed to the hard disk server. aof_fd =-1; // AOF file descriptor server. aof_selected_db =-1;/* Make sure the first time will not match */server. aof_flush_postponed_start = 0; // The last time the fsync was postponed to the hard disk server. aof_rewrite_incremental_fsync = REDIS_DEFAULT_AOF_REWRITE_INCREMENTAL_FSYNC; // whether incremental fsync is used for rewrite AOF files

The aof_current_size parameter is not initialized. Initialization of this value is initiated when the Server starts loading AOF file data.

Summary

This article briefly introduces the significance of parameters related to AOF mechanism implementation in Redis, so as to facilitate understanding of AOF data persistence mechanism. The implementation of AOF mechanism is complicated, it can be roughly divided into four parts: Server startup and AOF file data loading, append new data to AOF file, automatic rewrite AOF file, and client request BGREWRITEAOF command, the following blog will introduce the implementation of these three parts.