python log parser

The URL to access the Zabbix API is:http://x.x.x.x/zabbix/api_jsonrpc.phpx.x.x.x may be your IP or domain nameOverview of the Access process:1, first Login2. Zabbix server returns a token after successful authentication3, with this token to access a variety of data, do a variety of operations4, Complete!First, log in with RestclientIn the body of the JSON request, you have the following properties: jsonrpc-Version of the JSON-RPC protocol use

Label:execstripstatusimaref cursorllbcharsetble #!/usr/bin/env python# coding:utf-8# auther:liangkai# date:2018/6/26 11:26# License: (C) Copyright 2013-2017, Node Supply Chain Manager Corporation limited.# describe:import pymysqlimport reimport datetime# DB variablesdbhost = "192.168.189.18 5 "Dbport = 3306dbuser = ' root ' Dbpassword = ' 123456 ' charset = ' UTF8 ' # connection database conn_db = Pymysql.connect (host=dbhost,port=dbpo Rt,user=dbuser,

Tags: creating databases utf-8requestdbusmys file execution sys.argvcreate #!/usr/bin/env python # coding:utf-8 # auther:liangkai # date:2018/6/26 11:26 # License: (C) Copyright 2013-2018, Node Su pply Chain Manager Corporation Limited. # describe:import Pymysql import re import datetime import sys import time # DB variables dbhost = "192.168.189.185" Dbpo RT = 3306 Dbuser = ' root ' Dbpassword = ' ******** ' charset = ' utf8 ' log_file = sys.argv[1]

Tags: get open CTI count for close argv list src Because the database additions and deletions suddenly burst, need to query is the operation of those tables is particularly frequent, wrote a tool to analyze Bin-log, find and add to change the table, and the number of operations in descending order, the following is the code: 1 2 3 4 5 6 7 8 9 Ten One A - - the - - - + - + A at - - - - - in - to + - the * $

introduction logging module is a python built-in standard module, mainly used for output log, you can set the level of output logs, log save path, log files and rollback, etc. It can be said that the logging module consists mainly of 4 parts: *logger Logger , which provides interfaces that application code ca

# ! /Usr/bin/ENV Python # -*-Coding: UTF-8 -*- ''' This log class can output logs of different levels to different log files. ''' Import OS Import Sys Import Time Import Logging Import Inspecthandlers = {Logging. notset: " // Tmp/TNLOG-notset.log " , Logging. Debug: " // Tmp/TNLOG-debug.log " , Logging. Info: " // Tmp/TNLOG-info.log " ,

The python script collects ip address access times in log files for code sharing, Applicable log formats: 106.45.185.214 - - [06/Aug/2014:07:38:59 +0800] "GET / HTTP/1.0" 200 10 "-" "-"171.104.119.22 - - [06/Aug/2014:08:55:01 +0800] "GET / HTTP/1.0" 200 10 "-" "-"27.31.238.242 - - [06/Aug/2014:09:43:19 +0800] "GET / HTTP/1.0" 200 10 "-" "-"218.65.202.131 - - [06/

The method of writing a log is various, I this is I like the method, can make a referenceThere's nothing to say, just the code.Import TimedefWrite_log (value): Now_time= Time.time ()#get the current date and timeTime_format ='%y-%m-%d%h:%m:%s' #Specify the date and time formatTime_put = Time.strftime (time_format,now_time)#format time, time becomes YYYY-MM-DD HH:MI:SSfile_name ='/log/log.log'Log_file= Open

#导入正则模块import reauth="no_shutdown_"‘‘‘分析日志的模块，查找日志中标志性信息产生的次数‘‘‘#定义你需要查找的对象的正则表达式wordcheck#需要分析的日志的路径filesourcedef checklog(wordcheck,filesource): #定义一个空的字典用来存放查询的结果 size={} #异常捕获当文件不存在的时候抛出异常 try: #打开日志文件 file=open(filesource,"r") #循环读取日志文件的每一行 for i in file: #使用re模块的search功能查找当前行是否能和正则匹配 x=re.search(wordcheck,i) #如果匹配到结果则执行if中的代码 if x: #取出查询到的结果 tmp=x.group() #get函数作用是如果字典中取不到key的值则赋一个默认值，也就是每一次查询到一个新的结果就将这个结果作为key vlaue=0新加到字典中 size[tmp]=size.get(tmp,0) #在字典中将value加1，记录一次查找 size[tmp]+=1 #关闭文件

dictionary def Extract (line): pattern = ' ' (? POutput Result:{' request ': {' method ': ' GET ', ' protocol ': ' http/1.1 ', ' url ': '/o2o/media.html?menu=3 '}, ' remote_addr ': ' 183.60.212.153 ', ' datetime ': Datetime.datetime (2, 0, Max, Max, Max, Tzinfo=datetime.timezone (Datetime.timedelta), ' size ' ': 16691, ' status ': $, ' user_agent ': ' mozilla/5.0 (compatible; Easouspider; +http://www.easou.com/search/spider.html) '}　　ThreeRequest and DateTime processing functions are shortened

How to print error tracking: Method 1. Print to screen Try i = 0 Mydbhandler.executemany (SQL, TMP) Logging.info ("[Total cost Time:" + str (time.time ()-start) *1000) + "Ms]" Print "Total cost time:", (Time.time ()-start) *1000, "MS" Except EType, evalue, tracebackobj = Sys.exc_info () [: 3] print ' Type: ', etype Print "Value:", Evalue TRACEBACK.PRINT_TB (Tracebackobj) Method 2. Print to FileImport loggingLog_filename = '/tmp/logging_example.out 'Logging.basicconfig (filename=log_filename,l

are ordinary strings and Unicode strings, it is recommended to use Unicode strings!Because each + creates a new string, in order to avoid a lot of temporary strings, use the Join methodsl=["Hello", "we", "Love", "python"]SL = "". Join (SL) Print SLA single string is linked with a space as an intermediate symbol.Unicode encodingThe utf-8 is variable, one byte to four bytes, usually one kanji account of three bytes, or fourUtf-16 and utf-32, respective

,y=0): self.xpoint=xself.ypoint=yclass Retangle (object): def__init__ (SELF,POINT1,POINT2,POINT3,POINT4): self.point1=point1 self.point2=point2self.point3= point3self.point4=point4 Deffindck (SELF,POINT1,POINT2,POINT3,POINT4): self.length =abs (Point1.xpoint-point2.xpoint) self.width= abs (Point2.ypoint-point3.ypoint) defarea (self): self.finDCK (SELF.POINT1,SELF.POINT2,SELF.POINT3,SELF.POINT4) self.result=self.length*self.widthreturn Self.resultp1=point ( -1,2) p2=point (3,2) p3=point (3,-1) p

A simple Python log processing class Copy CodeThe code is as follows: #/usr/bin/python #coding =utf-8 Import Time,types Class Logsys: def __init__ (self, project, LogFileName = ' sys_log.txt '):Self.project = ProjectSelf.logfilename = LogFileName def get_log_time (self):Return Time.strftime ("%y-%m-%d%x", Time.localtime ()) def write2file (self, *formart):s =

less than 10 lines continue to 1000 reads, and so forth ~Specific implementation:#!/usr/bin/envpython#-*-coding:utf-8-*-"" "##authors:limanman# 51ctobg:http://xmdevops.blog.51cto.com/#purpose:# "" "From__future__importabsolute _import# Description: Import Public module importosimportsysimporttimeimportchardet# Description: Import other modules #if__name__== ' __main__ ':seekps=0; target= ' Data.txt ';try: # the pointer jumps to the end as the read position withopen ( target, ' r+b ') asf: F.se

processedPrint("The buns are finished .")defConsumer (name): whileTrue:ifQ.qsize () >0:Print("%s ate bun%s ....."%(Name,q.get ())) Q.task_done ()##告知这个任务执行完了Time.sleep (1) P=threading. Thread (target=producer,args= ("QJJ",)) C=threading. Thread (target=consumer,args= ("LSJ",)) P.start () C.start ( )"""qjj production of steamed bun 0...LSJ ate bun 0.....qjj production of steamed bun 1...LSJ ate bun 1.....qjj production of steamed bun 2...qjj production of steamed bun 3...lsj to eat the steamed b

in the regular expression into a backslash. The native string in Python solves this problem well, and the regular expression in this example can be expressed using R "\ \". Similarly, a "\\d" that matches a number can be written as r "\d". With the native string, you no longer have to worry about missing the backslash, and the expression is more intuitive.Re. I (re. IGNORECASE): Ignore case (full notation within parentheses, same as) M (MULTILINE): M

9 Month Thursday Today is a special day, always feel 9 months this day sounds strange. This morning, "Head first HTML and CSS" Chapter " table and more list ", not much content, read and do detailed notes and exercises. See Chapter "HTML form " Tomorrow, in fact the book is in total , chapter IV is " appendices and other " . The book will be finished by tomorrow and the next day. Also is to lay a foundation, continue to learn more advanced things to go. This afternoon read the"list" o

This morning did not concentrate on the study, I feel I may be tired, need to reduce the degree of learning to adjust a bit. These days the computer to make the faint sky, the body a little too unbearable. Time is a magical magic, and for half a month, every few days there is an idea to change strategy. The book I read this morning is Chapter 6 of Head first HTML and CSS , a primer onCSS . About half of it, there are more than 20 pages to see. What's the whole thing this morning? On the entire

of three nights of tossing, today successfully installed the Ubuntu operating system. Although there are many unhappy things in the middle that I did not think of, anyway, I can go to bed early tonight.Think about this half-month life, every day addicted to learning, I have not so dedicated my life to love to learn this thing. I've always had a problem where my eyes can see where I don't allow myself to be inferior to others. The hardest thing is that people know that I don't know. This strange