Rsync + inotify enables real-time server backup, rsyncinotify

Rsync + inotify enables real-time server backup, rsyncinotify

 

Inotify:

Inotify is integrated into the kernel. inotify is used as a third-party software to monitor directory changes through the interfaces provided by the kernel.

The inotifywait command can monitor fine-grained real-time changes to files in the directory. The monitored command is

Inotifywait-mrq -- timefmt '% d/% m/% y % H: % m' -- format' % T % w % f % e'-e modify, delete, create, attrib/var/web

Execute this command, and the listening will always be run on the command interface. When the directory changes, the command will output the changed content. Then, use rsync to synchronize directories.

Rsync uses two working modes for synchronization, one is based on the ssh service for synchronization, use method: rsync [option] src user @ host: dir

The second method is rsync Socket mode. Use the rsync service to synchronize rsync [option] src rsync: // user @ host/web.

Inotify + rsync monitors local data changes. The server is installed on the machine to be synchronized.

Several optimization application solutions of rsync

There are many solutions to solve the problem of rsync slowdown caused by increasing files.
1. Save fewer files in the source directory
This is a traditional optimization method, because although rsync synchronizes all files, it is the same as synchronizing recently updated files. Therefore, it deletes the directories on the source server, the number of files that have only been updated recently has become small and stable. Over time, the number of files will not increase rapidly. However, rsync cannot use the delete mode. to delete a file, you can empty it. If you have more strict requirements, you can use another program to delete it.
2. Use/dev/shm memory Partition
If the source directory keeps fewer files and the files do not exist on the hard disk and are put into the memory, the system IO problems can be avoided, however, this memory partition will lose all data after the system reboot. Although reboot is not often required, the risks also need to be calculated clearly.
3. Use the push Method
Because the performance problem occurs on the rsync client, if the source server is captured by the production server, the performance problem will occur on the production server, which is of course inappropriate. If rsync daemon is used on the production server, the source server executes the rsync command to push the file to the production server, and the performance problem is transferred to the source server, this ensures the stability of production servers to a certain extent.
4. Use only one instance for Synchronous comparison
If the files on the source server are to be synchronized to many production servers, rsync concurrency may occur. It can be analyzed that the files on these production servers are consistent at the same time. Therefore, it is a waste to compare each server with the source server. At this time, the source server and the production server can be synchronized once, and the-v parameter can be used to print out the log. Other production servers can avoid the comparison process several times by synchronizing the log file.
5. Use inotify
Inotify is not rsync, and inotify is a daemon process that can monitor file changes in the file directory and use rsync for file transmission based on the output, you can delete the file comparison link. Inotify is not complex to use. It monitors file changes in real time without consuming a lot of performance.
All of the above are optimized for rsync performance, but the optimization will also cause problems. In solutions 3, 4, and 5, if a production server has a server reboot due to load or other problems, synchronization fails during the reboot process, if this part of failed files are not processed, they will never be synchronized to the production server. In this case, multiple rsync methods can be used for processing. For example, inotify is used for real-time synchronization, and then complete rsync synchronization is performed hourly. This ensures a high synchronization speed and limits the risk of losing files within one hour.

How to synchronize two remote cent OS servers to a directory

Inotidy + rsync this solution can only achieve single-item synchronization. If you need two-way synchronization of directories, you can try inotidy + unison. I am currently using this solution, and make the two server directories into two-way synchronization.

App.yinxiang.com/..194f62
This is the note I wrote at the time. You can see the package included in it.