Redis data Persistence mechanism AOF analysis of the principle of configuration detailed

The source code referenced in this article is all from the Redis2.8.2 version.

The Redis configuration file is redis.conf, and the parsing code for the configuration file is shown in CONFIG.C

Original article reproduced please specify, this article from http://blog.csdn.net/acceptedxukai/article/details/18135219

Introduction to the implementation mechanism of AOF

AOF is append only file, implementation mechanism: Redis take a snapshot of the database, traverse all the databases, restore the data in the database to the protocol format of the instructions sent by the client, and then Redis create a temporary file to save these snapshot data, To the end of the snapshot program to modify the temporary file name to a normal aof file name, the original file is automatically discarded, due to the process of the snapshot may have new commands to modify the data in the database, the snapshot program after the end of the need to append the newly modified data to the AoF file, Subsequent commands from the client continue to be written to the disk according to different security levels. This supports real-time persistence, but it is possible to have data loss for a short period of time, which can be tolerated for general systems.

Configuration file redis.conf for aof parameter configuration explanation

appendonly

Meaning: Does Redis server turn on the AOF persistence mechanism

The CONFIG.C function loadserverconfigfromstring parse the code as

if (!strcasecmp (argv[0], "appendonly") && argc = = 2) {
	int yes;
	if (yes = Yesnotoi (argv[1)) = = =-1) {
		err = "argument must be ' yes ' or ' no '"; goto Loaderr;
	}
	Server.aof_state = yes? Redis_aof_on:redis_aof_off;
}

Appendfsync

Meaning: Redis policy to flush data from the OS data buffer to disk

# Appendfsync always fsync with newly added data
Appendfsync EVERYSEC Support Delay Fsync
# Appendfsync No no need fsync
Parsing code in CONFIG.C

if (!strcasecmp (argv[0], "Appendfsync") && argc = = 2) {//aof fsync policy
	if (!strcasecmp (argv[1], "no")) {
		Server.aof_fsync = Aof_fsync_no;
	} else if (!strcasecmp (argv[1], "always")) {
		server.aof_fsync = aof_fsync_always;
	} else if (!strcasecmp (argv[1), "Everysec")) {
		server.aof_fsync = aof_fsync_everysec;
	} else {
		err = "argument must be ' no ', ' always ' or ' ever Ysec ' ";
		Goto loaderr;
	}
}

No-appendfsync-on-rewrite

Meaning: When the rewrite aof child process or the RDB child process is executing, does the server support Fsync whether the data is flushed to the hard disk after the newly modified data is written to the AoF file

Parsing code in CONFIG.C

if (!strcasecmp (argv[0], "No-appendfsync-on-rewrite") && argc = = 2) {
	if (server.aof_no_fsync_on_rewrite = Yesnotoi (argv[1])) = =-1) {
		err = "argument must be ' yes ' or ' no '"; goto Loaderr;
	}
}

Server.aof_no_fsync_on_rewrite control the implementation code of the mechanism in the AOF.C flushappendonlyfile function, the specific code is

If Fsync is not supported, or if the AOF RDB child process is running, then return directly,
//But the data has been written to the aof file, just not flushed to the hard drive
if (Server.aof_no_fsync_on_rewrite & &
	(Server.aof_child_pid! =-1 | | Server.rdb_child_pid! = 1))
		return;

Appendfilename

Meaning: AoF's file name

Auto-aof-rewrite-percentage auto-aof-rewrite-min-size 64MB

These two configurations are not explained, see the comments on the AOF parameter in the REDISSERVER structure below

parameters of AOF in redisserver data structure

struct Redisserver {... */* AOF persistence */int aof_state; /* REDIS_AOF_ (on| Off|                  wait_rewrite) */int aof_fsync;             /* Kind of Fsync () policy */char *aof_filename;    /* Name of the AOF file */int aof_no_fsync_on_rewrite; /* Don ' t fsync if a rewrite is in prog.           */int AOF_REWRITE_PERC; /* Rewrite AOF if% growth is > M and ...     */off_t Aof_rewrite_min_size; /* The AOF file is at least N bytes.    */aof file minimum number of bytes off_t aof_rewrite_base_size; /* AOF size on latest startup or rewrite.         * * The size of the last aof file off_t aof_current_size; /* AOF current size.      * * The size of the current aof file int aof_rewrite_scheduled; /* Rewrite once BGSAVE terminates.            */pid_t Aof_child_pid;   /* PID If rewriting process */subprocess PID list *aof_rewrite_buf_blocks; /* Hold changes during an AOF rewrite.      */SDS AOF_BUF;       /* AOF buffer, written before entering the event loop */int aof_fd; /* File descriptor of currently selected AOF file */AOF document descriptor int aof_selected_db; /* Currently selected DB in AOF */time_t Aof_flush_postponed_start;            /* UNIX time of postponed AOF flush */time_t Aof_last_fsync;   /* UNIX time of Last Fsync () */time_t aof_rewrite_time_last; /* Time used by last AOF rewrite run.  */time_t Aof_rewrite_time_start; /* Current AOF rewrite start time.   */int aof_lastbgrewrite_status;  /* REDIS_OK or Redis_err */unsigned long aof_delayed_fsync; /* Delayed AOF Fsync () counter */int aof_rewrite_incremental_fsync;/* fsync incrementally while rewriting? */ ............ }

String aof_buf and linked list aof_rewrite_buf_blocks are important parameters for processing differential data after appending aof file with rewrite aof file respectively

initialization and annotation of aof parameters by Initserverconfig function

Server.aof_state = Redis_aof_off;  aof File Open Server.aof_fsync = Redis_default_aof_fsync; Fsync policy, default Fsync server.aof_no_fsync_on_rewrite per second = Redis_default_aof_no_fsync_on_rewrite;//aof Whether the RDB child process supports Fsync, that is, after writing the aof file, the data is flushed to the hard disk SERVER.AOF_REWRITE_PERC = REDIS_AOF_REWRITE_PERC; Auto rewrite increment value server.aof_rewrite_min_size = redis_aof_rewrite_min_size;   AoF file Minimum bytes server.aof_rewrite_base_size = 0;   Automatic rewrite calculates the cardinality of the aof file increment, equal to the number of bytes of the last aof file server.aof_rewrite_scheduled = 0;
Rewrite the task schedule, when the client sends the BGREWRITEAOF instruction, if the current rewrite child process is executing, the bgrewriteaof of the client request becomes a scheduled task, and the AOF child process is executed rewrite Server.aof_last_fsync = time (NULL);  Recent Fsync data to hard disk Time Server.aof_rewrite_time_last =-1; Rewrite duration server.aof_rewrite_time_start =-1; Rewrite start time server.aof_lastbgrewrite_status = REDIS_OK; State Server.aof_delayed_fsync after rewrite = 0;   Delay Fsync to hard disk server.aof_fd =-1; AOF file Descriptor server.aof_selected_db =-1; /* Make sure the first time would not match */server.aof_flush_postponed_start = 0; Last deferred FSYnc to hard disk Time Server.aof_rewrite_incremental_fsync = Redis_default_aof_rewrite_incremental_fsync;//rewrite AOF file with incremental Fsync

Where the aof_current_size parameter is uninitialized, the initialization of the value is initialized when the server starts loading aof file data.

Summary

This article is a brief introduction of Redis in the AOF mechanism to implement the relevant parameters of the meaning, convenient to the next AOF data persistence mechanism understanding, the implementation of AOF mechanism is more complex, roughly divided into the server boot load aof file data, append new data to the AoF file, Automatic rewrite aof file, client request bgrewriteaof Instructions four sections, the following blog will describe the implementation of these four sections.