Python scripts process log files in real time,

Python scripts process log files in real time,

This Python script is used to monitor the content of real-time files. For example, the Error or time out fields can be customized. It is my first real Python script and I feel bloated, but I plan to record it on the blog (for beginners, do not try it ), I really hope you can give me some advice (now the file size value is output to a file every time, and shell commands are nested in it, I think this can be optimized, but I still don't know how to do it). The alarm is based on zabbix, And the Custom template is executed once every S.

#! /Usr/local/bin/python3.5 ### Destription: reads log information in real time ### Author: Danny Deng ### Datetime: 2016-11-17import re, time, subprocess, OS, linecache ##### define the log file file_name = "/usr/local/nginx/logs/error. log "file_number ="/usr/local/zabbix_agent/number.txt "j = int (0) seek = int (0) # judgment process: whether the file exists --- determine whether the file storing the log size exists --- determine the size of number size and filesize ### define the function to read the file content by row def readline (): #### if, to determine whether seek is greater than 0, assign a value if it is greater than 0. Otherwise, it is initially 0 while True: ##### defines the file and reads each row based on the seek value, each time tell is assigned to seekwith open (file_name, 'R') as f: global seek # seek = seekf. seek (seek) data = f. readline () if data: seek = f. tell () yield dataelse: ###### convert the Python variable to the shell variable global file_numberos.environ ['seek '] = str (seek) OS. environ ['file _ number'] = str (file_number) OS. system ('echo $ seek> $ file_number ') OS. system ('chown zabbix. zabbix $ file_number ') returndef func_for (): j = int (0) for I in readline (): f_find = re. findall (r "check time out", I, flags = re. IGNORECASE) if "check time out" in f_find: j + = 1 ##### no output 0. if there is a value output, the second value that matches error appears. try: print (j) failed t NameError: print (int ("0") ### determine whether the log file exists if OS. path. isfile (file_name): ### determine whether the object with the size of the stored file exists if OS. path. isfile (file_number): #### the size of the file to be read if the file exists. Assign the value to seek_numberseek_number = int (linecache. getline (file_number, 1) #### determine the size of the stored file and the current file (determine whether the file is regenerated) if OS. path. getsize (file_name)> = seek_number and seek_number> 0: seek = seek_numberfunc_for () ### if it is a new file, seek is assigned 0 else: # open (arg1, "a + "). write ("0") # seek = int (linecache. getline (file_number, 1) seek = int (0) func_for () #### file_number does not exist, create a new one, and assign the seek variable to 0 else: # open (file_number, "a + "). write ("0") # seek = int (linecache. getline (file_number, 1) OS. environ ['file _ number'] = str (file_number) OS. system ('echo 0> $ file_number ') OS. system ('chown zabbix. zabbix $ file_number ') func_for () else: print ("Error") quit ()

The above is a small series of Python scripts introduced to you to process log File scripts in real time, I hope to help you, if you have any questions, please leave a message, the small series will reply to you in a timely manner. Thank you very much for your support for the help House website!