Python implements real-time monitoring files

In the case of high business stability requirements, operations to be able to detect problems in a timely manner, sometimes need to do real-time analysis of the application log, when a condition is met immediately alarm, rather than passively wait for the problem to solve, such as to monitor Nginx $request_time and $UPSTREAM_ Response_time time, analysis of the most time-consuming request, and then to improve the code, the log will be in real-time analysis, the discovery of long-time statements to the police out, reminding developers to pay attention to, of course, this is one of the application scenarios, This monitoring method can also be applied to any need to judge or analyze the file, so today we will look at how to implement real-time monitoring files in Python, I give three methods of the example:

The first type:

This is the simplest and easiest to understand because everyone knows that Linux has a tail command, so you can directly use the Popen () function to invoke this command to execute the fetch output, as follows:

Logfile= ' Access.log ' command= ' tail-f ' +logfile+ ' |grep ' timeout ' ' popen=subprocess. Popen (command,stdout=subprocess. Pipe,stderr=subprocess. Pipe,shell=true) while True:line=popen.stdout.readline (). Strip () Print line

The second type:

Using Python to operate the file, with the file object's tell (), the Seek () method to get the current file location and to move to the location, the code is as follows:

Import timefile = open (' Access.log ') while 1:where = File.tell () line = File.readline () if not line:time.sleep (1) File.seek ( where) Else:print line,

The third type:

Using the yield of Python to implement a generator function, and then call this generator function, so that the log file changes when the new line is printed, the code is as follows:

Import timedef Follow (thefile): Thefile.seek (0,2) while true:line = Thefile.readline () if not line:time.sleep (0.1) Continueyield Lineif __name__ = = ' __main__ ': logfile = open ("Access-log", "r") Loglines = Follow (logfile) for line in Logline S:print Line,

Finally, we explain the use of the Seek () function, which receives 2 parameters: File.seek (off, whence=0), moving the off action marker (file pointer) from the file, positive movement toward the end direction, and negative numbers moving in the beginning direction. If the whence parameter is set, the starting bit is set to whence, 0 is the starting point, 1 is the current position, and 2 represents the end of the file.

The above is the three common methods, the specific log analysis code can be based on their own business logic to achieve, complete.

The above is the whole content of this article, I hope that everyone's learning has helped, but also hope that we have a lot of support topic.alibabacloud.com.