Python scripts process log files in real time,

Source: Internet
Author: User

Python scripts process log files in real time,

This Python script is used to monitor the content of real-time files. For example, the Error or time out fields can be customized. It is my first real Python script and I feel bloated, but I plan to record it on the blog (for beginners, do not try it ), I really hope you can give me some advice (now the file size value is output to a file every time, and shell commands are nested in it, I think this can be optimized, but I still don't know how to do it). The alarm is based on zabbix, And the Custom template is executed once every S.

#! /Usr/local/bin/python3.5 ### Destription: reads log information in real time ### Author: Danny Deng ### Datetime: 2016-11-17import re, time, subprocess, OS, linecache ##### define the log file file_name = "/usr/local/nginx/logs/error. log "file_number ="/usr/local/zabbix_agent/number.txt "j = int (0) seek = int (0) # judgment process: whether the file exists --- determine whether the file storing the log size exists --- determine the size of number size and filesize ### define the function to read the file content by row def readline (): #### if, to determine whether seek is greater than 0, assign a value if it is greater than 0. Otherwise, it is initially 0 while True: ##### defines the file and reads each row based on the seek value, each time tell is assigned to seekwith open (file_name, 'R') as f: global seek # seek = seekf. seek (seek) data = f. readline () if data: seek = f. tell () yield dataelse: ###### convert the Python variable to the shell variable global file_numberos.environ ['seek '] = str (seek) OS. environ ['file _ number'] = str (file_number) OS. system ('echo $ seek> $ file_number ') OS. system ('chown zabbix. zabbix $ file_number ') returndef func_for (): j = int (0) for I in readline (): f_find = re. findall (r "check time out", I, flags = re. IGNORECASE) if "check time out" in f_find: j + = 1 ##### no output 0. if there is a value output, the second value that matches error appears. try: print (j) failed t NameError: print (int ("0") ### determine whether the log file exists if OS. path. isfile (file_name): ### determine whether the object with the size of the stored file exists if OS. path. isfile (file_number): #### the size of the file to be read if the file exists. Assign the value to seek_numberseek_number = int (linecache. getline (file_number, 1) #### determine the size of the stored file and the current file (determine whether the file is regenerated) if OS. path. getsize (file_name)> = seek_number and seek_number> 0: seek = seek_numberfunc_for () ### if it is a new file, seek is assigned 0 else: # open (arg1, "a + "). write ("0") # seek = int (linecache. getline (file_number, 1) seek = int (0) func_for () #### file_number does not exist, create a new one, and assign the seek variable to 0 else: # open (file_number, "a + "). write ("0") # seek = int (linecache. getline (file_number, 1) OS. environ ['file _ number'] = str (file_number) OS. system ('echo 0> $ file_number ') OS. system ('chown zabbix. zabbix $ file_number ') func_for () else: print ("Error") quit ()

The above is a small series of Python scripts introduced to you to process log File scripts in real time, I hope to help you, if you have any questions, please leave a message, the small series will reply to you in a timely manner. Thank you very much for your support for the help House website!

Related Article

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: info-contact@alibabacloud.com and provide relevant evidence. A staff member will contact you within 5 working days.

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.