Use Python to implement a simple project monitoring _python

An interface system in a company that is primarily a system interface to a third party, so the system will interact with projects in many other companies. Then a very painful problem, so many companies interface, the stability of different company interface is very large, access to a large amount of time, some not how the interface on all kinds of errors.

This interface system has just been developed soon, the entire system, in the comparative edge of the position, unlike other projects, there is a log library, and SMS alarm, once the problem, in many cases are user feedback back, so, my idea is, pick up Python, for this project to write a monitoring. If in the process of invoking a Third-party interface, a large number of errors, indicating that the interface has a problem, you can take action faster.

The project is also a log library, all the Info,error log is every minute to scan the storage, log library is used in MySQL, table has several particularly important fields:

• Level log Levels
• Message Log Contents
• file_name Java code file
• Log_time Log Time

There is a log library, you do not have to go to the online Environment scan log analysis, directly from the log library to start. Because the log library is swept every 1 minutes on the line, then I go to the log library every 2 minutes sweep, if swept to a certain number of error log on the alarm, if only one or two errors can be ignored, that is, a short time a large number of error logs, you can determine the system has a problem. Alarm mode is used to send mail, so you need to do the following several things:
1. Operate MySQL.
2. Send mail.
3. Scheduled Tasks.
4. Log.
5. Run the script.

Clear up a few things, you can do it.
manipulating Databases

Using MYSQLDB This driver, the direct operation of the database, mainly is the query operation.
To get a connection to a database:

Def get_con ():
 host = "127.0.0.1"
 port = 3306
 logsdb = "Logsdb"
 user = "root"
 password = "Never tell Y ou "
 con = MySQLdb.connect (Host=host, User=user, Passwd=password, Db=logsdb, Port=port, charset=" UTF8 ")
 return Con

Get the data from the log library, get the data 2 minutes before the current time, and start by calculating the time according to the current time. Before, the calculation has a problem and has now been modified.

Def calculate_time (): Now
 
 = Time.mktime (DateTime.Now (). Timetuple ()) -60*2 result
 = Time.strftime ('%y-%m-%d %h:%m:%s ', Time.localtime (now)
 

Then, log Library query data based on time and log levels

Def get_data ():
 select_time = Calculate_time ()
 logger.info ("Select time: +select_time)
 sql =" Select File _name,message from Logsdb.app_logs_record "\
   " where Log_time > "+" "+select_time+" ' "\
   " and level= "+" ERROR ' "\
   " ORDER by log_time desc "
 conn = Get_con ()
 
 cursor = conn.cursor ()
 cursor.execute (SQL)
 Results = Cursor.fetchall () cursor.close () Conn.close () return
 
 results

Send mail

Using Python to send messages is simpler, using the standard library Smtplib
Here use 163 mailbox to send, you can use other mailbox or Enterprise mailbox all line, but host and port to set correct.

def send_email (content):
 
sender = "sender_monitor@163.com"
receiver = ["rec01@163.com", "rec02@163.com"]
host = ' smtp.163.com '
port = 465
msg = mimetext (content)
msg[' from '] = "sender_monitor@163.com"
Msg[' to '] = "rec01@163.com,rec02@163.com"
msg[' Subject '] = "System error Warning"
 
try:
smtp = Smtplib. Smtp_ssl (host, port)
smtp.login (sender, ' 123456 ')
smtp.sendmail (sender, receiver, msg.as_string ())
Logger.info ("Send email success")
except Exception, E:
logger.error (E)

Timed tasks

Use a separate thread, scan every 2 minutes, and send an email notification if the error level has more than 5 log bars.

Def task (): While
True:
logger.info ("Monitor Running")
 
results = Get_data ()
If results are not None Len (Results) > 5:
content = "Recharge error:"
logger.info ("A lot of error,so send mail") for
R in re Sults:
content + = r[1]+ ' \ n '
send_email (content) sleep
(2*60)

Log

Configure the log log.py for this small script so that the logs can be exported to the file and console.

# coding=utf-8
Import logging
 
logger = Logging.getlogger (' MyLogger ')
logger.setlevel (logging. DEBUG)
 
fh = logging. Filehandler (' Monitor.log ')
fh.setlevel (logging.info)
 
ch = logging. Streamhandler ()
ch.setlevel (logging.info)
 
formatter = logging. Formatter ('% (asctime) s-% (name) s-% (levelname) s-% (message) s ')
Fh.setformatter (Formatter)
Ch.setformatter (Formatter)
 
Logger.addhandler (FH)
logger.addhandler (CH)

So, finally, this little monitor program is like this app_monitor.py

# Coding=utf-8 Import Threading Import MySQLdb from datetime import datetime import time import smtplib from email.mime.t Ext import mimetext from log import Logger def get_con (): host = "127.0.0.1" port = 3306 Logsdb = "logsdb" user = "Root" password = "never tell you" con = MySQLdb.connect (Host=host, User=user, Passwd=password, Db=logsdb, Port=port, ch arset= "UTF8") return con def calculate_time (): now = Time.mktime (DateTime.Now (). Timetuple ()) -60*2 result = Time.s Trftime ('%y-%m-%d%h:%m:%s ', Time.localtime (now) return to result def get_data (): Select_time = Calculate_time () logg Er.info ("Select time: +select_time) sql =" Select file_name,message from Logsdb.app_logs_record "\" where Log_time &G t; " 
 + "'" +select_time+ "'" \ "and level=" + "' ERROR '" "ORDER BY log_time DESC" conn = Get_con () cursor = Conn.cursor () Cursor.execute (SQL) results = Cursor.fetchall () cursor.close () Conn.close () return results def Send_email (CO ntent): Sender = "sender_monitor@163.com "receiver = [" rec01@163.com "," rec02@163.com "] host = ' smtp.163.com ' port = 465 msg = Mimetext ( Content) msg[' from '] = "sender_monitor@163.com" msg[' to '] = "rec01@163.com,rec02@163.com" msg[' Subject '] = "System Erro R warning "TRY:SMTP = smtplib.  Smtp_ssl (host, port) smtp.login (sender, ' 123456 ') smtp.sendmail (sender, receiver, msg.as_string ()) Logger.info ("Send Email success ") except Exception, E:logger.error (e) def task (): While True:logger.info (" Monitor running ") r  Esults = Get_data () If results is not None and Len (results) > 5:content = "Recharge Error:" Logger.info ("a lot of error,so Send Mail ") for R in Results:content + + + r[1]+ ' \ n ' send_email (content) Time.sleep (2*60) def R Un_monitor (): monitor = threading.
 Thread (Target=task) Monitor.start () if __name__ = = "__main__": Run_monitor ()

Run the script

Scripts run on the server and are managed using supervisor.
Install supervisor on the server (CENTOS6), and then add the configuration in/etc/supervisor.conf:

Copy Code code as follows:

[Program:app-monitor]
Command = python/root/monitor/app_monitor.py
Directory =/root/monitor
user = root

Then run Supervisord boot supervisor in the terminal.
Run Supervisorctl in the terminal, go to the shell, run status to view the running state of the script.
Summary

This small monitoring idea is very clear, you can continue to modify, such as: monitoring the specific interface, send SMS notifications and so on.
Because there is a log library, less to the online official environment scan log trouble, so, if there is no log library, you should be on the online environment scan, in the official online environment must be careful wow ~