Python uses the Xlsxwriter template to complete the nginx trend map and automatically email

Objective:

Came to the new company, the leader issued a demand: a weekly inspection of the server, as well as analysis of the ngxin trend, because the company's actual production environment server is not particularly many, consider the temporary first to complete the needs of the script.

General idea: Collect data--write Excel made chart--Send mail

I. Collection of NGINX data

The main is to do an analysis of nginx log, statistical data. The company uses the system comes with the logrotate cut Nginx log, only save 10 days log, and compressed, using shell analysis will appear two days, so: Copy the log files to other paths, decompression, and analysis, the script is as follows:

#!/bin/bashlogname=nginx.access.log*                #日志文件名LogPath =/var/log/nginx/nginx.access.log*     #日志路径NginxPath =/data/scripts /nginxlog/          #解压日志路径Filename =/data/scripts/nginxlog/ nginx.log     #数据收集文件Temporary =/data/scripts/nginxlog/temporary    #一个临时文件 # Copy files and unzip ' cp  $LogPath   $NginxPath '               ' gunzip   $NginxPath $logname ' #由于日志有两个日期, compared to egg pain, the following output is: 13/may/2016 xxxxfunction nglog () {        for i in  ' awk  ' gsub (/\[/, " ", $) '  $1|awk -F  ":| "   ' {++s[$5]}end {for  (i in s)  print i} '         do        num= ' grep   "$i"  $1|awk  "{print $1} ' |sort|uniq -c |wc -l '         echo $ i  $num         done} #目录下所有日志进行分析for  i in  ' ls $ Nginxpath|grep nginx.access.log ' do    nglog  $NginxPath $i >> $ temporarydone# Remove 2016, enter into log acc= ' awk  ' {a[$1]+=$2}end{for (i in a) print i,a[i]} '  $ temporary|sort|sed  ' s/\/[0-9]\{4\}//g ' echo   "$ACC"  >  $Filename # Delete temporary files and log files rm  -rf  $Temporaryrm  -rf  $NginxPath $logname

This collects the logs, so how do you keep the old data and the new data?

Write another script in the root directory, change the name to the old data file name before each execution

' MV $DataFile $LastData '

Second, the chart is made

Generated using Xlsxwriter, some of the code is as follows:

#!/usr/bin/python# -*- coding: utf-8 -*-import xlsxwriterimport timeimport  Linecachedatafile= '/data/scripts/serverdata ' lastfile= '/data/scripts/lastdata ' # # #处理nginx日志new_data  =  Ngdata (datafile) last_data = ngdata (lastfile) new_item=[]new_value=[]last_item=[]last_value=[]for  i in new_data:    new_item.append (I.split () [0])     new _value.append (int (I.split () [1]) For l in last_data:    last_item.append ( L.split () [0])     last_value.append (int (L.split () [1]) # # #创建excel表格Excel_name  =  " Server_%s.xlsx " %  (Time.strftime ("%y-%m-%d ", Time.localtime ()) Workbook = xlsxwriter. Workbook (Excel_name) Worksheet = workbook.add_worksheet () # # #项目的样式ItemStyle  = workbook.add_ Format ({         ' Font_size ':10,          ' bold ': true,         ' bg_color ': ' #101010 ',          ' Font_color ': ' #FEFEFE ',         ' align ': ' Center ',          ' align ': ' vcenter ',         ' top ': 2,          ' left ':2,         ' Right ':2,         ' bottom ': 2} # # #目标值的样式ValueStyle  =  Workbook.add_format ({         ' font_size ':9,          ' font_color ': ' #151515 ',         ' align ': ' Center ',         ' align ': ' Vcenter ',          ' top ':2,         ' left ':2,          ' right ': 2,  &NBsp;      ' Bottom ': 2} # # #写入nginx   Data row2 = 69col2 = 0col3  = 0row3 = 74col4 = 0col5 = 0worksheet.insert_textbox (' D68 ', ' Nginx_New_Data ' {' Fill ': {' none ': True}, ' width ': ' max ', ' height ': ' +} ' worksheet.insert_textbox (' D73 ', ' nginx_last_data ', {' Fill ': {') None ': True}, ' width ': max, ' height ': +}) for nt in new_item:    worksheet.write (Row2,col2,nt,itemstyle)     col2 += 1for nv in new_value:     worksheet.write (Row2+1,col3,nv,valuestyle)     col3 += 1for  lt in last_item:    worksheet.write (Row3,col4,lt,itemstyle)      col4 +=1for lv in last_value:    worksheet.write (Row3+1, Col5,lv,valuestyle)     col5 +=1## #Nginx图表NginxChart  = workbook.add_chart ({' Type ': ' line '}) #定义最新数据的线条Color et nginxchart.add_series ({     ' categories ': ' =sheet1! $A $: $I $ ',          ' values ': ' =sheet1! $A $71: $I $71 ',              ' data_labels ': {' value ':true},         ' name ': ' New '  data ',         ' line ':{              ' color '  :  ' #4A4AFF ',              ' width ': 2,    }, #定义旧数据线条颜色等NginxChart. Add_series ({      ' values ': ' =sheet1! $A $76: $I $76 ',         ' name ': ' last  Data ',         ' line ':{              ' color '  :  ' #BEBEBE ',              ' Dash_type ': ' Dash_dot ',             ' width ':1,    &NBSP (},}) Nginxchart.set_y_axis ({' name ': ' Nginx '}) Worksheet.insert_chart (' A78 ', Nginxchart) Workbook.close ()

：

650) this.width=650; "src=" Http://s3.51cto.com/wyfs02/M01/80/0E/wKiom1c1im7hGi5UAAJ4euCgycE420.jpg "title=" 111. JPG "alt=" wkiom1c1im7hgi5uaaj4eucgyce420.jpg "/>

Third, send the mail

After generating Excel, send a message with some of the following code:

Import smtplibfrom email.mime.text import mimetextfrom email. Mimemultipart import mimemultipartfrom email. mimebase import mimebasesender =  ' email address ' receiver = [' sender list ']subject =  ' Server inspection (%s) '  % (Time.strftime ("%y.%m.%d", Time.localtime ())) smtpserver =  ' Smtp.163.com ' username =  ' username ' password =  ' password ' msg = mimemultipart (' alternative ') msg[' Subject '] = subjecthtml =  "" "" Message Body "" Part = mimetext (HTML, ' HTML ', ' Utf-8 ')    #! This place to specify the code, otherwise the mail will appear garbled Msg.attach (part) Att = mimetext (open (Excel_name, ' RB '). Read (), ' base64 ', ' Utf-8 ') att[ "Content-type"] =  ' Application/octet-stream ' att["content-disposition"] =  "attachment; Filename=server.xlsx ' Msg.attach (att) smtp = smtplib. SMTP () smtp.connect (' smtp.163.com ') smtp.login (Username,password) Smtp.sendmail (sender,receiver,msg.as_string ()) Smtp.quit ()

At last

Add the script to the task schedule and customize the time execution.

Attach the final packing effect:

650) this.width=650; "src=" http://s2.51cto.com/wyfs02/M00/80/0B/wKioL1c1jO2jPaUjAAI1y6z0F3Y058.jpg "title=" 2222. JPG "alt=" wkiol1c1jo2jpaujaai1y6z0f3y058.jpg "/>650) this.width=650; src=" Http://s3.51cto.com/wyfs02/M00/80/0E /wkiom1c1ji_xfgcgaalx98h2yh0847.jpg "title=" 3333.jpg "alt=" Wkiom1c1ji_xfgcgaalx98h2yh0847.jpg "/>

This article is from the "Cat Demon Blog" blog, please be sure to keep this source http://maoyao.blog.51cto.com/8253780/1773074

Python uses the Xlsxwriter template to complete the nginx trend map and automatically email