INotify file system event monitoring rsync in conjunction with

Each run rsync command will traverse the target directory, when the file is not long, this is no problem, once the file number to a certain scale, then each traversal will consume a lot of resources, if only to the frequent changes in the directory to synchronize, you can also ignore this problem, if each change of the directory more large, Then you need to use INotify, INotify is a powerful, fine-grained, asynchronous file system event monitoring mechanism, the Linux kernel from 2.6.13, joined the support of INotify, through INotify can monitor the file system to add, delete, modify, Mobile and other events, but inotify only provides the C language interface, not convenient to call, so we need to install the first Inotify-tools

The following from:

http://www.ibm.com/developerworks/cn/linux/l-inotifynew/

inotify-The file system change notification mechanism in the Linux 2.6 kernel

This paper introduces in detail the new notification mechanism of file system change introduced in Linux 2.6.13 kernel inotify, and illustrates its use and typical application cases. first, the introduction

As we all know, Linux desktop systems with a lot of unsatisfactory than MAC or Windows, in order to improve this situation, the open source community to the user state requires the kernel to provide some mechanism so that user state can be in time to learn what the kernel or underlying hardware device, so as to better manage the device, To provide users with better services, such as HotPlug, Udev and INotify is the birth of this demand. Hotplug is a kind of kernel to the user State application to notify the mechanism of some incidents of hot-swappable devices, the desktop system can use it to effectively manage the equipment, udev dynamically maintain/dev under the device files, INotify is a file system change notification mechanism, such as file increase, Delete and other events can immediately let user state that the mechanism is well-known desktop search engine project Beagle introduced, and in Gamin and other projects were applied.

In fact, there is a similar mechanism known as dnotify before INotify, but it has many drawbacks:

1. For each directory that you want to monitor, users need to open a file descriptor, so if you need to monitor more directories, it will cause many file descriptors to open, especially if the monitored directories on mobile media (such as CD and USB disks) will not umount these file systems because Dnotify application opens the file descriptor in use of the file system.

2. Dnotify is a directory-based, it can only get directory change events, of course, in the directory changes in the file will affect the directory to raise the directory change events, but to know which file changes through directory events, you need to cache many stat structure of data.

3. The Dnotify interface is very unfriendly and it uses signal.

Inotify is designed to replace dnotify, overcoming dnotify flaws and providing a better, concise and powerful notification mechanism for file changes:

1. Inotify does not need to open a file descriptor for the target being monitored, and if the target is being monitored on removable media, the watch that corresponds to the monitored target will be automatically deleted and a Umount event will occur after the file system is umount on the media.

2. Inotify can either monitor files or monitor directories.

3. Inotify uses system calls rather than Sigio to notify file system events.

4. Inotify uses file descriptors as an interface, so you can monitor file system changes using the usual file I/O operations Select and poll.

Inotify can monitor file system events include: In_access, that is, files are accessed in_modify, files are write in_attrib, file attributes are modified, such as chmod, chown, touch, etc. in_close_write, Writable files are close in_close_nowrite, can not write files by close in_open, files are OPEN in_moved_from, files are moved, such as MV In_moved_to, files are moved, such as MV, CP In_create, To create a new file In_delete, the file is deleted, such as RM in_delete_self, since delete, that is, an executable file deletes itself in_move_self on execution, since the move, that is, an executable file that moves itself when executed In_unmount, the host file system is Umount in_close, file is closed, equivalent to (In_close_write | In_close_nowrite) In_move, files are moved, equivalent to (In_moved_from | IN_MOVED_TO)

Note: The file mentioned above also includes a directory.

Back to page first two, user interface

In user state, INotify is used through three system calls and file i/operations on the returned file descriptor, the first step in using INotify is to create a inotify instance:

                int fd = Inotify_init ();

Each inotify instance corresponds to a separate sorted queue.

File system change events are called an object management of watches, each watch is a two-tuple (target, event mask), the target can be a file or directory, the event mask represents the INotify event that the application wishes to focus on, and each bit corresponds to a inotify event. The Watch object is referenced by the Watch descriptor, watches by the path name of the file or directory. The directory watches will return events that occurred above all the files in the directory.

The following function is used to add a watch:

                int wd = Inotify_add_watch (fd, path, mask);

FD is the file descriptor returned by Inotify_init (), path is the pathname of the target being monitored (that is, the file name or directory name), mask is the event mask, and each represented event is defined in the header file Linux/inotify.h. You can modify the event mask in the same way, that is, to change the INotify event that you want to be notified. Wd is a watch descriptor.

The following function is used to delete a watch:

        int ret = Inotify_rm_watch (FD, WD);

FD is the file descriptor returned by Inotify_init (), and WD is the watch descriptor returned by Inotify_add_watch (). A Ret is the return value of a function.

A file event is represented by a inotify_event structure that is obtained by using the normal file read function read by the file descriptor returned by Inotify_init ()

:

struct Inotify_event {
        __s32           wd;             /* Watch Descriptor * *
        __u32           mask;           /* Watch Mask * *
        __u32           cookies;         * * Cookies to synchronize two events * *
        __u32           len;            /* Length (including nulls) of name */
        char            name[0];        /* Stub for possible name *
/};

The WD in the structure is the watch descriptor for the monitored target, mask is the event mask, Len is the length of the name string, the name is the path name of the target being monitored, the name field of the structure is a pile, it is just for the user to refer to the filename, the filename is longer. It is actually immediately following the structure, and the filename will be filled with 0 so that the next event structure can be aligned 4 bytes. Note that Len also counts the number of bytes populated.

You can get multiple events at once by using the read call, provided the BUF is large enough.

                size_t len = Read (FD, buf, Buf_len);

BUF is an array pointer to a inotify_event structure, Buf_len specifies the total length to read, the BUF size is at least less than Buf_len, and the number of events returned by the call depends on the Buf_len and the length of the file name in the event. Len is the actual number of bytes read, that is, the total length of the event obtained.

You can use Select () or poll () on the file descriptor FD returned by Function inotify_init (), or you can use the IOCTL command fionread on FD to get the length of the current queue. Close (FD) deletes all watch added to the FD and makes the necessary cleanup.

                int inotify_init (void);
        int inotify_add_watch (int fd, const char *path, __u32 mask);
        int inotify_rm_watch (int fd, __u32 mask);

Back to the first three, kernel implementation mechanism

In the kernel, each inotify instance corresponds to a inotify_device structure:

struct Inotify_device {wait_queue_head_t wq;            /* Wait queue for I/O struct IDR IDR;            * IDR mapping WD-> Watch * * struct semaphore sem;         * * Protects this bad boy * * struct list_head events;        /* List of queued events * * struct list_head watches;          /* List of Watches * * atomic_t count;          /* Reference count */struct user_struct *user;     /* User who opened this dev */unsigned int queue_size;    /* Size of the queue (bytes) */unsigned int event_count;     /* Number of pending events * * unsigned int max_events;        /* Maximum number of events */U32 LAST_WD; /* The last WD allocated */};

The WQ is the waiting queue, and the process that is blocked by the read call hangs on the wait queue, and IDR maps the watch descriptor to the corresponding Inotify_watch,sem to synchronize access to the struct, which is the list of events that occur on the INotify instance, which is All events monitored by the otify instance are inserted into the list after the occurrence, watches is the list of watch that is monitored for the inotify instance, and Inotify_add_watch inserts the newly added watch into the list, count is the reference count, and the user is used to trace Describes the user who created the INotify instance, queue_size represents the number of bytes in the event queue for the INotify instance, Event_count is the number of events in the event list, max_events is the maximum allowable number of events, LAST_WD is the last assigned Wat The CH descriptor.

Each watch corresponds to a inotify_watch structure:

struct Inotify_watch {
        struct list_head        d_list;/* entry in Inotify_device ' s list */struct        list_head i_ List /* entry in Inode ' s list
        *                /atomic_t count;  /* Reference count *
        /struct inotify_device   *dev;   * * associated device
        /struct inode            *inode/* Associated inode                     /S32 wd;     /* Watch Descriptor * *
        u32                     mask;   /* Event mask for this watch *
/};

D_list point to the list of all Inotify_device, i_list points to a list of all monitored inode, count is a reference count, and dev points to the inotify structure that corresponds to the Inotify_device instance where the watch is located , the inode points to the watch to be monitored Inode,wd is assigned to the watch descriptor, mask is the Watch event mask, indicating which file system events it is interested in.

The struct Inotify_device is created when the user state invokes Inotify_init () and is disposed when the file descriptor returned by Inotify_init () is closed. The struct Inotify_watch is created when the user state invokes the Inotify_add_watch () and is freed when the user state invokes Inotify_rm_watch () or Close (FD).

The inotify system adds two fields to the inode structure, whether it is a directory or a file, which corresponds to an inode structure in the kernel:

#ifdef config_inotify
	struct list_head	inotify_watches/* Watches on this inode */
	struct semaphore	Inotify_sem;	/* Protects the Watches list * * *
#endif

Inotify_watches is a watch list on a monitored target, and whenever the user invokes Inotify_add_watch (), the kernel creates a inotify_watch structure for the added watch and inserts it into the inode for the target being monitored. List of inotify_watches. Inotify_sem is used to synchronize access to the Inotify_watches list. When one of the events mentioned in the first part of the file system occurs, the corresponding file system code displays the call fsnotify_* to report the corresponding event to the inotify system, where the * is the corresponding event name, and the current implementation includes: Fsnotify_move, Files are moved from one directory to another directory fsnotify_nameremove, files are removed from the directory fsnotify_inoderemove, Fsnotify_create are deleted, new files are created Fsnotify_mkdir, new directories are created Fsnotify_access, files are read fsnotify_modify, files are written fsnotify_open, files are opened fsnotify_close, files are closed fsnotify_xattr, file extension attributes are modified Fsnotify_change, the file was modified or the original data was modified

An exception is Inotify_unmount_inodes, which is called when the file system is umount to notify the Umount event to the inotify system.

The notification function mentioned above finally calls Inotify_inode_queue_event (Inotify_unmount_inodes directly calls Inotify_dev_queue_event), The function first determines whether the corresponding inode is monitored, which is achieved by looking at whether the Inotify_watches list is empty, and if the inode is not monitored and nothing is done, return immediately, instead, traverse the inotify_watches list, See if the current file action event is monitored by some watch, if it is, call inotify_dev_queue_event, otherwise, return. The function Inotify_dev_queue_event first determines whether the event is a duplicate of the previous event, and if it is discarded and returned, otherwise it determines whether the event queue of the INotify instance, Inotify_device, overflows, if it overflows, Generates an overflow event that otherwise generates a current file action event that is built by Kernel_event, Kernel_event creates a inotify_kernel_event structure, and then inserts the structure into the corresponding inotify_ Device Events list, and then wakes up waiting queues that await wq in the inotify_device structure. The user-state process that wants to monitor file system events is on the wait queue Wq when the INotify instance (that is, the file descriptor returned by Inotify_init () is invoked, but when there are no events.

Back to top four, using the example

Here is an example of using INotify to monitor file system events:

#include <linux/unistd.h> #include <linux/inotify.h> #include <errno.h> _syscall0 (int, inotify_init ) _syscall3 (int, inotify_add_watch, int, FD, const char *, PATH, __u32, mask) _syscall2 (int, inotify_rm_watch, int, FD, __
U32, Mask) char * monitored_files[] = {"./tmp_file", "./tmp_dir", "/mnt/sda3/windows_file"};
	struct Wd_name {int wd;
char * name;
};
#define Wd_num 3 struct wd_name wd_array[wd_num]; char * event_array[] = {"File is accessed", "File was modified", "file attributes were changed", "Writtable file Clo Sed "," unwrittable file Closed "," file was opened "," file is moved from X "," file is moved to Y "," Subfile is Crea Ted, "Subfile was deleted", "Self is deleted", "Self was moved", "", "Backing FS is unmounted", "Event queued ove
Rflowed "," File was ignored "};
	#define EVENT_NUM #define MAX_BUF_SIZE 1024 int main (void) {int fd;
	int WD;
	Char buffer[1024];
	char * offset = NULL;
	struct Inotify_event * Event; int Len, TMP_len;
	Char strbuf[16];
	
	int i = 0;
	FD = Inotify_init ();
		if (FD < 0) {printf ("Fail to initialize inotify.\n");
	Exit (-1);
		For (i=0 i<wd_num; i++) {wd_array[i].name = monitored_files[i];
		WD = Inotify_add_watch (FD, Wd_array[i].name, in_all_events);
			if (WD < 0) {printf ("Can ' t add watch for%s.\n", wd_array[i].name);
		Exit (-1);
	} WD_ARRAY[I].WD = WD;
		while (len = read (fd, buffer, max_buf_size)) {offset = buffer;
		printf ("Some event happens, Len =%d.\n", Len);
		event = (struct inotify_event *) buffer;
			while ((char *) event-buffer) < Len) {if (Event->mask & In_isdir) {memcpy (strbuf, "Direcotory", 11);
			else {memcpy (strbuf, "File", 5);
			printf ("Object type:%s\n", strbuf);
				For (i=0 i<wd_num; i++) {if (event->wd!= wd_array[i].wd) continue;
				printf ("Object Name:%s\n", wd_array[i].name);
			Break
			printf ("Event Mask:%08x\n", event->mask);
	For (i=0 i<event_num; i++) {			if (event_array[i][0] = = ' Continue ');
				if (Event->mask & (1<<i)) {printf ("Event:%s\n", Event_array[i]);
			} tmp_len = sizeof (struct inotify_event) + event->len; 
			event = (struct inotify_event *) (offset + tmp_len);
		Offset + + Tmp_len; }
	}
}

The program monitors all file system events that occur in the current directory tmp_file with the directory Tmp_dir in the current directory, and it also monitors file system events that occur on the file/mnt/sda3/windows_file, noting that/mnt/sda3 is The hanging point of SATA hard drive partition 3.

The attentive reader may notice that the program header uses _SYSCALLN to declare inotify system calls, because these system calls were introduced in the latest stable kernel 2.6.13 and glibc did not implement the library function version of these system calls, so In order to be able to use these system calls in a program, these new systems must be declared through _syscalln, where N is the actual number of arguments to be declared for the system call. It is also important to note that the header file of the system must match the kernel being started, and in order for the above program to compile successfully, the 2.6.13 kernel header file (including include/linux/*, include/asm/*, and Include/asm-generic /*) is in the header file search path and is the first priority search header file path because _syscalln needs to use the linux/unistd.h and asm/unistd.h in these header files, which contain the system call number of the INotify three system calls __nr_ Inotify_init, __nr_inotify_add_watch and __nr_inotify_rm_watch.

Therefore, to successfully compile this program, simply copy the user's compiled kernel header file to the path of the program, and compile it using the following command: