(iv) Persistence---Configuration and principles of aof persistence

AOF persistence and aof rewrite configuration:

default The AOF mode is closed, such as:

650) this.width=650; "Src=" Http://s3.51cto.com/wyfs02/M02/83/FA/wKiom1eCXvyjrKL6AAFSGLyzA5Q846.jpg-wh_500x0-wm_3 -wmp_4-s_509066519.jpg "title=" 1.jpg "alt=" Wkiom1ecxvyjrkl6aafsglyza5q846.jpg-wh_50 "/>

If you want to turn it on, just rewrite No to Yes. such as:

650) this.width=650; "Src=" Http://s2.51cto.com/wyfs02/M00/83/F9/wKioL1eCXwSTC5H8AAAkOy29djo700.jpg-wh_500x0-wm_3 -wmp_4-s_2584521239.jpg "title=" 2.jpg "alt=" Wkiol1ecxwstc5h8aaakoy29djo700.jpg-wh_50 "/>

The default file name is appendonly.aof, and you can also modify the file name. The default save directory is also a setting in the Dir configuration item in the configuration file, which shares a directory with the RDB. such as:

650) this.width=650; "Src=" Http://s1.51cto.com/wyfs02/M00/83/FA/wKiom1eCXwzzL43_AAAgurUvs48363.jpg-wh_500x0-wm_3 -wmp_4-s_2526173145.jpg "title=" 3.jpg "alt=" Wkiom1ecxwzzl43_aaaguruvs48363.jpg-wh_50 "/>

The default synchronization policy is per second, such as:

650) this.width=650; "Src=" Http://s5.51cto.com/wyfs02/M01/83/F9/wKioL1eCXx3BezMBAAAheXW2VM4565.jpg-wh_500x0-wm_3 -wmp_4-s_3958770979.jpg "title=" 4.jpg "alt=" Wkiol1ecxx3bezmbaaahexw2vm4565.jpg-wh_50 "/>

Let's do some work on the database and then look at the contents of the appendonly.aof file

650) this.width=650; "Src=" Http://s5.51cto.com/wyfs02/M01/83/FA/wKiom1eCXyWjnzxMAAAtqM_jRwI519.jpg-wh_500x0-wm_3 -wmp_4-s_5363823.jpg "title=" 5.jpg "alt=" Wkiom1ecxywjnzxmaaatqm_jrwi519.jpg-wh_50 "/>

It records all write operations.

* *	Represents 2 parameters
$6	Indicates that the first parameter has a length of 6
SELECT	First parameter
$	The second parameter has a length of 1
0	A second parameter

AoF overriding policy

650) this.width=650; "Src=" Http://s1.51cto.com/wyfs02/M01/83/F9/wKioL1eCX-mD8Dx5AAFihO2iDMI666.jpg-wh_500x0-wm_3 -wmp_4-s_768535899.jpg "title=" 6.jpg "alt=" Wkiol1ecx-md8dx5aafiho2idmi666.jpg-wh_50 "/>

AOF Persistence Implementation principle:

when After the AOF is persisted, the Update command is appended to the end of the Aof_buf buffer and then written to the AOF file by the buffer when an update is made to the database .

the content recorded in the AOF file is an instruction to the data update operation. The document itself is recorded in text, such as :

650) this.width=650; "Src=" Http://s5.51cto.com/wyfs02/M01/83/FA/wKiom1eCYBnzWFM3AAAtqM_jRwI354.jpg-wh_500x0-wm_3 -wmp_4-s_628753289.jpg "title=" 5.jpg "alt=" Wkiom1ecybnzwfm3aaatqm_jrwi354.jpg-wh_50 "/>

when data needs to be recovered, the execution The update instructions that are recorded in the AOF file can be completed. Artificial look inside the instructions, and then manually knock command can also be completed.

AoF overriding implementation principle:

because AOF Persistence is a way to preserve the state of a database by logging commands, and over time AOF files will certainly grow and, if unchecked, will have an impact on AOF persistence performance and data recovery. The following examples illustrate the need to rewrite more graphically:

Let's take a compression list for example

650) this.width=650; "Src=" Http://s5.51cto.com/wyfs02/M02/83/FA/wKiom1eCYFfg-vTCAAB-iEt0rmQ375.jpg-wh_500x0-wm_3 -wmp_4-s_3591693912.jpg "title=" 7.jpg "alt=" Wkiom1ecyffg-vtcaab-iet0rmq375.jpg-wh_50 "/>

according to AOF principle, then the above red box 5 commands are appended to the AOF file, in fact, we see the last list of the state is the bcdef value. That is to say, to achieve the final state of the example, 5 additional commands are required . So in a lot of memory read and write business AOF file growth quickly, for example to solve this problem,Redis provides AOF rewrite functionality.

AOF Rewrite is to create a new AOF file to replace the existing AOF file, in fact AOF overrides do not operate on existing old AOF files.

list file, write a rpushlist B C D E f command to avoid writing 5

The AOF rewrite program aof_rewrite function to complete the task of creating a new AOF file, but the function is not called directly by the Redis master process, but instead is executed using the child process daemon ( Bgrewriteaof, the command is actually executing aof_rewrite, which is called by a child process, and the main process is not blocked, then the parent process can continue to provide external responses during the rewrite. The whole process is as follows:

• when the override is triggered, the parent process calls a function that creates a child process to execute Bgrewriteaof, the child process creates a temporary file, and the parent process continues to provide read-write services to the external

• sub-process traversal database, Output the latest state of each key value to a temporary file, in bgrewriteaof aof Buffer and AOF rewrite buffer ( aof_rewrite_buf_blocks Buffer ( aof_buf aof buffer contents are not recommended to synchronize to an existing aof

• AOF The child process notifies the parent process after the rewrite is complete, the parent process calls the signal handler function

• The signal processing function blocks the parent process from providing read and write operations (in a very short time, without blocking and with inconsistent data), and then AOF the contents of the rewrite buffer to the new AOF file, and finally replaces the existing AOF file with the new AOF file (rename operation)

Appendfsync Option Description:

Parameters	Description
Always	will be Aof_buf all content in the buffer is written and synchronized to the AOF file, execute immediately Write () and the Fsync () system calls. The security of the data is highest, but the execution is slowest, and if a failure only loses the contents of an event loop.
Everysec	will beAof_bufall the contents of the buffer are written to theAOFfile, if the last synchronizationAOFtime distance of this time than1seconds, the synchronization is performed, once every secondWrite ()and theFsync ()system calls. Data security is centered, execution is fast and only lost1seconds of data.
no	will aof_buf all contents of buffer write To file, but when synchronization is determined by the operating system, only system call. The write action is efficient, but does not perform synchronization, but a single synchronization consumes the longest time and data security is the lowest, losing all data after the last synchronization.

Here's a special note Linux system of file writing and synchronization principle ' no parameter, if you put always understanding is always, always, or in real time, and the everysec understanding for every second, that no aof

in the When Redis calls the Appendfsync function, it actually calls a write () function before it calls sync () or fsync () . function (for any program that wants to write data to disk, the process is the same, with some exceptions).

User space: Normal process area, user initiated, this area of code does not directly access the hardware

Kernel space: operating system area, can communicate with device controller

when the Write () function is called, once the function returns a normal value, we may think that the data has been written to the disk, but in fact, the operating system in the implementation of the disk file io , in order to ensure the efficiency of Io, A dedicated address space is used in memory, the space is called kernel space, and within the kernel space there will be a data buffer for IO (this buffer and the previously said aof_buf buffer is not a concept, although all in memory), The function of the write () function is to write data to the IO buffer in the kernel space.

650) this.width=650; "src=" http://s1.51cto.com/wyfs02/M00/83/F9/wKioL1eCYaiDhKUNAABora4npp4231.jpg "title=" 8.JPG " alt= "Wkiol1ecyaidhkunaabora4npp4231.jpg"/>

Kernel-Space The IO Buffer also has a certain size, and when the buffer is not full or there is no synchronization cycle, the data passed by the write () function is continuously written to the buffer, and when the buffer is full or a synchronization period is reached, The contents of the buffer will be submitted to the output queue, when the data is required to reach the queue first, the real disk IO operation is performed, the data is written to disk (although it is used to write to the disk, but the real action is not the move but the copy, after the completion of the kernel space IO The buffer does not release the space occupied by the data). This approach is called deferred write.

so there is a problem, when the write () function is called does not mean that the data is really saved to the disk, but there will be an illusion, that is, when you request the file again, you can display the last time you update the content, In fact, this content is not read from the disk, but from the user space buffer read. Then the problem mentioned above, if the data in the kernel space in the IO buffer, and at this time the operating system failure, power outages and other anomalies will result in loss of data.

to address the problem of data loss, the Unix system provides three functions for sync, Fsync, and fdatasync .

Function	Function
Sync	function returns 0 indicates success, the function is responsible for putting all kernel space IO the modified content in the buffer is pushed to the input queue and then returned, and it does not wait for all disks IO The operation is complete. So even if the sync function is called, it is not a successful save to disk.
Fsync	function returns 0 means success, with sync different, it only takes effect on a single file of the specified file descriptor, forcing all modified data connected to the file to be transferred to disk, And wait for disk io 0 write () fsync ()
fdatasy NC	fsync inode . Span style= "line-height:107%;font-family: ' The song Body '; > information. So relative to fsync

So if you look at the contents, you know Appendfsync in No the meaning of the parameter.

This article is from the "Little Demon's Home" blog, so be sure to keep this source http://littledevil.blog.51cto.com/9445436/1822967

(iv) Persistence---Configuration and principles of aof persistence