Python-based apahce website log analysis example

This article mainly introduces the example of implementing log analysis for apahce website using python. if you need it, you can refer to the example of maintaining the script. it is written in disorder, just as an example, demonstrate how to quickly use the tool to quickly achieve the goal:
Application: shell and python data interaction, data capturing, and code conversion

The code is as follows:

# Coding: UTF-8
#! /Usr/bin/python
'''
Program Description: apache access. log analysis
Analyze the source of the website IP address
Date:
Author: gyh9711

Program Description: Application to: shell and python data interaction, data capturing, encoding conversion
'''
Import OS
Import json
Import httplib
Import codecs

LogFile = '/var/log/apache2/access. log'
# Logs
LogMess = '/tmp/acc. log'
If OS. path. isfile (logMess ):
OS. system ('CP/dev/null % s' % logMess)
File = codecs. open (logMess, 'W + ', encoding = 'utf-8 ')

Def cmd (cmd ):
Return OS. popen (cmd). readlines ()
'''
Def getIp (ip ):
Return json. loads (OS. popen ("/usr/bin/curl http://ip.taobao.com/service/getIpInfo.php? Ip = % s "% ip). readline () ['data']
'''
Conn = httplib. HTTPConnection ('IP .taobao.com ')
Def getIpCountry (ip ):
Conn. request ('GET', '/service/getIpInfo. php? Ip = % s' % ip)
R1 = conn. getresponse ()
If r1.status = 200:
Return json. loads (r1.read () ['data']
Else:
Return "Error"
# Analyze the access. log file and convert it to a python array
File. write (u "field description: ip access times data ip country City isp number province region \ n ")
IpDb = []
For I in cmd ('''/usr/bin/awk '{print $1}' % s | sort | uniq-c ''' % LogFile ):
Ip = I. strip (). split ('')
IpDb. append (ip)
# Analyze IP Address sources through APIs provided by taobao
For I in ipDb:
_ TmpD = getIpCountry (I [1])
# Format Description: ip address access data region where the isp number of the ip address country city is located
Out = "% s" % (I [1]. ljust (20), I [0]. ljust (10), _ tmpD ['country']. ljust (20), _ tmpD ['city']. ljust (16), _ tmpD ['isp _ id']. ljust (16), _ tmpD ['region']. ljust (16), _ tmpD ['region']. ljust (16 ))
Print out
File. write ("% s \ n" % out)

Conn. close ()
File. close ()

'''

'''