Real-time file backup using Linux commands-incron, rsync, and inotify

Real-time file backup using Linux commands-incron, rsync, and inotify

I,Rsync, incronIntroduction

For details about the rsync tool and its use with the cron tool for Scheduled backup, refer to the blog: http://blog.csdn.net/wangjun2008/article/details/4268539;

Inotify is a file change notification mechanism, which is introduced in Linux Kernel 2.6.13;

Inotify is just an API that needs to be called by developing an application;

Inotify-tools is an implementation of inotify. It is a set of components, including a C library and several command line tools, these command line tools can be used to monitor the events of a file system through command lines or scripts;

The name incron is the combination of inotify cron system, which means a cron system Based on inotify. It contains a background daemon (incrond) and an event Editor (incron );

For more information, see: http://www.howtoforge.com/triggering-commands-on-file-or-directory-changes-with-incron

II,Verify if the Linux kernel supports inotify (skipped)

# Check whether the machine kernel supports inotify (multiple methods)

# Method 1: Check the Linux kernel version. if the version is 2.6.13 +, the kernel supports inotify.

$ Uname-r

# Method 2: Check whether the/usr/include/sys/inotify. h file exists. If yes, the kernel supports inotify.

$ Ls-l/usr/include/sys | grep inotify. h

# Method 3: Check whether the/proc/sys/fs/inotify directory exists and whether the directory contains the following files. If yes, the kernel supports inotify.

$ Ls-l/proc/sys/fs/inotify

-Rw-r -- 1 root 0 Oct 9 09: 36 max_queued_events

-Rw-r -- 1 root 0 Oct 9 09: 36 max_user_instances

-Rw-r -- 1 root 0 Oct 9 09: 36 max_user_watches

# Note: the files in the/proc/sys/fs/inotify directory are default Kernel Parameters of inotify.

Max_queued_eventsDefault Value: 16384;

When inotify_init is called, the maximum number of events that can be queued in the inotify instance is allocated. The event exceeding this value is discarded, but the IN_Q_OVERFLOW event is triggered;

Note: max_queued_events is the maximum length of Inotify-managed queues. The more frequently the file system changes, the larger the value;

If Event Queue Overflow is displayed in the log, it indicates that max_queued_events is too small. You need to adjust the parameters and use them again;

Max_user_instancesDefault Value: 128;

Specifies the maximum number of inotify instatnces that can be created by each user ID;

Max_user_watchesDefault Value: 8192;

Specifies the maximum number of watches associated with each inotify instance (the maximum number of files or directories that can be monitored );

3.,Incron and rsyncDownload and install

For rsync download and installation, see http://blog.csdn.net/wangjun2008/article/details/4108539;

Incron installation environment requirements: Linux Kernel 2.6.13 +, which can be viewed through uname-;
Incron: http://inotify.aiken.cz /? Section = incron & page = download & lang = en;

Incron installation steps:

$ Download the incron-0.5.10.tar.gz package;

$ Tar zxvf incron-0.5.10.tar.gz
$ Incron-0.5.10 cd
$ Make
$ Make install
# After the installation is complete, the binary files incrond and incrontab will be obtained and stored in the system PATH, which can be directly used;

# Use yum to install: yum install incron

# Use sudo to install: sudo apt-get install incron

Iv. incrond and incrontab configuration, usage, and parameter description

# Start the background daemon thread

Incrond [start]

# View incrond help

Incrond-h

Usage: incrond [<options>] <Operation> may be one of the following: These options may be used: -?, -- About gives short information about program -H, -- help prints this help text -N, -- foreground runs on foreground (no daemonizing) -K, -- kill terminates running instance of incrond -F <FILE>, -- config = <FILE> overrides default configuration file (requires root privileges) -V, -- version prints program version

# View incron help

Incrontab-h

Usage: incrontab [<options>] <operation> Incrontab [<options>] <FILE-TO-IMPORT> <Operation> may be one of the following: -?, -- About gives short information about program -H, -- help prints this help text -L, -- list to view the user's incron monitoring -R, -- remove removes user table -E, -- edit Add/edit user incron monitoring -T, -- types: view the monitoring event types supported by incron -D, -- reload request incrond to reload user table -V, -- version prints program version   These options may be used: -U <USER>, -- user = <USER> overrides current user (root permission required) -F <FILE>, -- config = <FILE> overrides default configuration file (root permission required)

Among them, the most important options are-l,-t,-e;-e, which is used to configure incron event monitoring. The format is as follows:

<Path> <event> <command>

Each metric occupies one row. The default value is/var/spool/incron;

[Option description]
<Path>Files or directories to be monitored;

<Event>Object events. Multiple events are separated by commas (,). The following monitoring events can be used:

IN_ACCESS: The file is accessed;

IN_MODIFY: The file is modified;
IN_ATTRIB: file attributes are modified, such as chmod, chown, and touch;
IN_CLOSE_WRITE: The writable file is closed;

IN_CLOSE_NOWRITE: The file that cannot be written is disabled;
IN_CLOSE: The file is closed, equivalent to (IN_CLOSE_WRITE | IN_CLOSE_NOWRITE );

IN_OPEN: The file is opened;
IN_MOVED_FROM: The file is removed, such as mv;
IN_MOVED_TO: The file is moved, such as mv and cp;
IN_MOVE: The file is moved, equivalent to (IN_MOVED_FROM | IN_MOVED_TO );

IN_CREATE: Create a new file;
IN_DELETE: The file is deleted, such as rm;
IN_DELETE_SELF: indicates that an executable file is deleted when it is executed;
IN_MOVE_SELF: self-moving, that is, an executable file moves itself during execution;
IN_ONESHOT: Only one event is monitored;

IN_ONLYDIR: only monitors directories;

IN_UNMOUNT: the host file system is umount;

IN_ALL_EVENTS: all of the above events;

IN_DONT_FOLLOW: Don't dereference pathname if it is a symbolic link;

<Command>System commands or scripts (the test found that the 'redirection 'in the command is invalid and can only be used in the script). You can also use the following variables:
$ @: Indicates <path>, that is, the Monitored object;

$ #: File/directory in which a system event occurs (for example, if a file under a folder is monitored and changes, $ # indicates the file name );
$ %: Indicates the <event> event (event code );
$ &: Indicates the event (event number );

[Example]

Start incrond: incrond start;

Configure incrontab: incrontab-e. The content is/home/test/mon IN_ALL_EVENTS echo "$ @# #$ %"

Go to the/home/test/mon/directory. If you operate on the directory, the corresponding event information is printed;

[This part is reproduced from: http://hi.baidu.com/qu3999352/item/ad1aae6381e2329cc4d2496f]

V,InCronReal-time data backup with rsync

(Omitted)

Sat.,[Extension] AboutInotifywait, inotifywatch commandIntroductionAnd usage

(1)Inotify-tools download and Installation

Download inotify-tools: https://github.com/rvoicilas/inotify-tools/wiki

$ Tar zxvf inotify-tools-3.14.tar.gz
$ Inotify-tools-3.14 cd
$./Configure

$ Make
$ Make install
# View inotify commands

$ Ls-l/usr/local/bin | grep inotify
-Rwxr-xr-x 1 root 37264 04-14 13:42/usr/local/bin/inotifywait
-Rwxr-xr-x 1 root 35438 04-14 13:42/usr/local/bin/inotifywatch

(2) Use of inotifywait and inotifywatch commands and parameter descriptions

After inotify-tools is installed according to the preceding method, the inotifywait and inotifywatch commands are generated in the/usr/local/bin directory;

Inotifywait: used to wait for a specific event;
Inotifywatch: used to collect statistics on the monitored file system, including the number of times each inotify event occurs;

Common options for inotifywait commands are as follows:

-M, -- monitor for continuous monitoring. inotifywait exits after a specified file event is monitored by default;

-R, -- recursive recursively monitors all subdirectories and their files under a specified directory. If the number of files in the directory to be monitored is large, you usually need to modify the max_users_watchs Kernel Parameter because the default value is 8192;

-E <event>, -- event <event> specifies the specific event to be monitored. All file events are monitored by default;

-- Timefmt <fmt> When % T is used in the -- format option, the -- timefrt option can be used to specify a custom time format that complies with the strftime specification, available time format characters can be obtained through the strftime manual page; -- common parameters after timefrt are '% d/% m/% y % H: % m ';

-- Format <fmt> customizes the output format of inotifywait, for example, -- format '% T % w % F'. Common format characters are as follows:

% W: display the name of the monitored file;

% F: if the object of an event is a directory, the name of the monitored directory is displayed. The default value is an empty string;

% T: Use the custom time format in the -- timefmt option;

-Fromfile reads the files to be monitored or excluded from the files. One file contains one row and the excluded files start;

-Z: the rows and columns of the "-zero" output table, even if the element is empty

-R: "-recursive" monitors all subdirectories in a directory.

-T: "-timeout" to set the timeout time

-E: "-event" only listens to specified events

-Q, -- quiet does not output monitoring results;

When you want to exclude synchronizing a directory, add the -- exculde = PATTERN parameter for rsync (path is relative path );

Add the -- exclude or -- excludei parameter to inotifywait when you want to exclude the processing of event monitoring in all directories;

For example, to monitor the create, delete, modify, close_write events in the/tmp/test directory and all its internal files, run the following command:

# Inotify-r -- timefmt '% d/% m/% y % H: % m' -- format' % T % w % F'-e create, delete, modify, close_write/tmp/test

Note: This part of content is reproduced from: http://blog.chinaunix.net/uid-233544-id-3129307.html