Python implements real-time monitoring files

This article mainly introduces three methods for implementing real-time monitoring files in python. if you are interested, you can refer to the situation where the business stability requirements are high, in order to detect problems in a timely manner, O & M personnel sometimes need to analyze application logs in real time. when a condition is met, an alarm is triggered immediately, instead of waiting for the problem to be solved, for example, to monitor the $ request_time and $ upstream_response_time of nginx, analyze the most time-consuming requests, and then improve the code, then analyze the logs in real time, an alarm is triggered when long statements are found, reminding developers to pay attention to them. of course, this is one of the application scenarios. this monitoring method can also be applied to any location where files need to be judged or analyzed, so today, let's take a look at how to use python to implement real-time monitoring files. I will give three method instances ::

First:

This is the simplest and easy to understand. as we all know that there is a tail command in linux, you can directly use the Popen () function to call this command to obtain the output. the code is as follows:

logfile='access.log'command='tail -f ‘+logfile+'|grep “timeout”‘popen=subprocess.Popen(command,stdout=subprocess.PIPE,stderr=subprocess.PIPE,shell=True)while True:line=popen.stdout.readline().strip()print line

Second:

Python is used to perform file operations. The tell () and seek () methods of the file object are used to obtain the current file location and the location to be moved, respectively. the code is as follows:

import timefile = open(‘access.log')while 1:where = file.tell()line = file.readline()if not line:time.sleep(1)file.seek(where)else:print line,

Third:

Use yield of python to implement a generator function, and then call this generator function to print new lines when the log file changes. The code is as follows:

import timedef follow(thefile):thefile.seek(0,2)while True:line = thefile.readline()if not line:time.sleep(0.1)continueyield lineif __name__ == ‘__main__':logfile = open(“access-log”,”r”)loglines = follow(logfile)for line in loglines:print line,

Finally, I will explain the usage of the seek () function. this function receives two parameters: file. seek (off, whence = 0), move the off Operation mark (file pointer) from the file, move the positive number to the end direction, and move the negative number to the start direction. If the whence parameter is set, the start position set by whence prevails. 0 indicates starting from scratch, 1 indicates the current position, and 2 indicates the end position of the file.

The above are three common methods. the specific log analysis code can be implemented and completed based on your own business logic.

The above is all of the content in this article. I hope it will be helpful for everyone's learning and support for PHP's Chinese web.

For more python-related articles on real-time monitoring files, refer to the Chinese PHP website!