Inotify: Efficient, real-time Linux file system event monitoring framework

Inotify: An efficient, real-time Linux file System event Monitoring Framework Overview-Why do I need to monitor the file system?

In daily work, people often need to know that there are changes in some files (clips), such as:

• Notification configuration file changes
• Track changes to some critical system files
• Monitor the overall usage of a partition disk
• Automatic cleanup when the system crashes
• Automatically triggers the backup process
• Notifies the server when a file is uploaded at the end

The notification mechanism for file polling is typically used, but this mechanism applies only to files that are frequently changed (because it ensures that I/O is available every x seconds), is otherwise very inefficient, and sometimes loses some type of change, such as when the file's modification time has not changed. Data integrity systems such as Tripwire, which track file changes based on time scheduling, can do nothing if they want to monitor file changes in real time. INotify was born like this. This article will give a brief introduction to INotify, tell us how to monitor the folders, report related message events as soon as they change, and introduce some related tools that we can add to our own toolbox.

What the hell is inotify?

INotify is a notification mechanism for file changes, and the Linux kernel is introduced from 2.6.13 onwards. The more famous kqueue in BSD and Mac OS Systems is the ability to efficiently track changes in Linux file systems in real time. In recent years, with Fsnotify as the backend, almost all major Linux distributions support the inotify mechanism. How do you know if your Linux kernel supports the inotify mechanism? Very simply, execute the following command:

% grep inotify_user/boot/config-$ (uname-r) config_inotify_user=y

If the output (' config_inotify_user=y '), then you can enjoy the INOTIFY tour right away.

Simple File change Notification sample:

The good start is half the success, for understanding the inotify mechanism, let's start with the Inotifywait program, which is included in the Inotify-tools Toolkit. If we're going to monitor the operation on the/srv/test folder, just do it:

% Inotifywait-rme modify,attrib,move,close_write,create,delete,delete_self/srv/testsetting up watches.  Beware:since-r is given, this could take a while! Watches established.

While the above tasks are running, we'll do the following in another shell: Create a folder, create a file under a new folder, and then delete the newly created file:

% mkdir/srv/test/infoq% echo TODO >/srv/test/infoq/article.txt% rm/srv/test/infoq/article.txt

The following information will be printed in the shell running inotifywait:

/srv/test/create,isdir infoq/srv/test/infoq/create article.txt/srv/test/infoq/modify article.txt/srv/test/infoq/ Close_write,close Article.txt/srv/test/infoq/delete Article.txt

It is obvious that we will receive the notification whenever there is a change. For more information on events provided by inotify (such as modify, Atrrib, etc.), please refer to the manpage of Inotifywatch. In practice, if you do not want to monitor a large folder, you can use the inotifywait exclude option. For example: We want to ignore the folder/srv/test/large, then we can set up the monitoring:

% inotifywait--exclude ' ^/srv/test/(large|ignore)/'-rme modify,attrib,move,close_write,create,delete,delete_self/ Srv/testsetting up watches.  Beware:since-r is given, this could take a while! Watches established.

In the example above, we used a regular expression in the matching string of the exclude option because we did not want to exclude the file containing large or ignore in the name. We can test it:

% echo Test >/srv/test/action.txt% echo Test >/srv/test/large/no_action.txt% echo Test >/srv/test/ignore/no_ac tion.txt% echo Test >/srv/test/large-name-but-action.txt

Here inotifywait should only report the changes of ' action.txt ' and ' Large-name-but-action.txt ' two files, ignoring the files under subfolders ' Large ' and ' ignore ', and the result is true;

/srv/test/create action.txt/srv/test/modify Action.txt/srv/test/close_write,close action.txt/srv/test/create Large-name-but-action.txt/srv/test/modify Large-name-but-action.txt/srv/test/close_write,close Large-name-but-action.txt

In addition, by using the-t option, we can also define the monitoring time of the inotifywait, either to allow it to execute for a period of time or to keep it running. Util-linux-ng's logger command can also do this, but first send the relevant message events to the syslog and then analyze the integration from the Syslog server.

Inotifywatch-Use INotify to count file system access information

Inotify-tools also has a tool called Inotifywatch, which listens to the file system's message events and then counts the message events for each monitored file or folder, and then outputs the statistics. For example, we want to know what's on a folder:

% INOTIFYWATCH-V-e access-e modify-t 120-r ~/infoqestablishing watches ... Setting up Watch (es) On/home/mika/infoqok,/home/mika/infoq are now being watched. Total of watches. Finished establishing watches, now collecting statistics. Would listen for events for Seconds.total  modify  filename2      2       /home/mika/infoq/inotify/

Obviously, we are monitoring the ~/infoq folder here, and we can see that there are two events on the/home/mika/infoq/inotify. The method is simple but effective.

Configuration options for INotify

When using inotify, pay special attention to the two configurations in the kernel about it. First,/proc/sys/fs/inotify/max_user_instances specifies the upper limit of inotify instances each user can create, followed by/proc/sys/fs/inotify/max_user_ Watches specifies that each inotify instance can have a maximum of several monitoring (watch) connections. You can easily experiment to reach the upper limit during the run, such as:

% inotifywait-r/setting up watches.  Beware:since-r is given, this could take a while! Failed to watch/; Upper limit on inotify watches reached! Please increase the amount of inotify watches allowed per user via '/proc/sys/fs/inotify/max_user_watches '.

If you want to change these configurations, simply write a new value to the appropriate file as follows:

# cat/proc/sys/fs/inotify/max_user_watches8192# echo 16000 >/proc/sys/fs/inotify/max_user_watches# cat/proc/sys/ fs/inotify/max_user_watches16000

Some tools for using inotify

In recent times there have been many inotify-based tools, such as Incron, a cron-like daemon (daemon), where the traditional cron daemon is executed for a specified period of time, and Incron because of the use of inotify, Execution can be triggered by an event. At the same time, the installation of Incron is simple and intuitive, such as on Debian, first add users using Incron in/etc/incron.allow (Debian does not allow users to use Incron by default, because if incron use inadvertently, for example, to form a dead loop, will cause system downtime):

# echo Username >/etc/incron.allow

Then call "Incrontab-e", in the popup editor insert our own rules, such as the following simple rule, the file changes Incron will send us an email notification:

/srv/test/in_close_write mail-s "[Email protected]/$#\n" Root

From now on, once the files in the/src/test folder are modified, a message is sent. But be careful not to let Incron monitor the entire subtree, because INotify only focuses on inodes, not the files or folders, so inotify-based software needs to handle/prevent recursion. Please refer to Incrontab's manpage for details on Incontab use.

If you also want to process the incoming folder, you may need to inoticoming. Inoticoming performs certain actions when a file enters the Incoming folder, allowing the inoticoming to be used to manage the Debian repository (for example, once an upload source package or a newly added binary package is automatically compiled in the repositories), and It can also be used to monitor whether a system has new uploads, and if so, to send notifications. Similar tools are available (they all have expertise): Inosync (folder synchronization service based on the message notification mechanism), iwatch (inotify based program, real-time monitoring of the file system), and LSYNCD (a daemon (daemon), Synchronize the local folder with Rsync).

INotify has even improved on traditional UNIX tools, such as tail. With the Inotail, with the-f option, you can replace the practice of polling files per second. In addition, GNU Coreutils has supported inotify from version 7.5, and we can run the following command to confirm:

# strace-e Inotify_init,inotify_add_watch tail-f ~log/syslog[...] Inotify_init ()                          = 4inotify_add_watch (4, "/var/log/syslog", in_modify| in_attrib| In_delete_self| in_move_self) = 1

From now on, the method of polling to make sure that the file needs to be re-read should be an antique.

Using INotify in Scripts

The inotify mechanism is not limited to tools, and in scripting languages you can also enjoy inotify, such as the use of pyinotify and Inotifyx in Python, The ruby version of Filesys-notify-simple and Linux-inotify2,inotify in Perl has Ruby-inotifyrb-inoty and FSSM.

Summarize

In summary, INotify provides Linux with a set of mechanisms for efficiently monitoring and tracking file changes that can be processed, debugged, and monitored in real time, and polling is a delay mechanism. For system administrators, INotify provides powerful support for implementing event-driven services such as system backup, build services, and program debugging based on file operations.

View English text: Inotify:efficient, real-time Linux File System Event Monitoring

Inotify: Efficient, real-time Linux file system event monitoring framework