Python parsing nginx Log ip,url,status

The Python script is as follows:

#!/usr/bin/env python#_*_coding:utf-8 _*___author__ =  ' Lvnian ' #!/usr/bin env  python# coding: utf-8import mysqldb as mysqlimport sys, osdb =  Mysql.connect (user= "root", passwd= "[email protected]", db= "Intest", host= "192.168.10.12")  # Database connection Information Db.autocommit (True) cur = db.cursor () cur.execute (' Set names utf8 ') dict_list =  {}with open (' Access.log ')  as logfin:    for line in  Logfin:        arr = line.split ('   ')          #  Get ip url  and status         ip = arr[0]        url = arr[6]         status = arr[8]        #  ip url  status as key, eachMeter +1        dict_list[(ip,url,status)] = dict_list.get (IP, Url,status), 0) +1    #  converted to list     ip_list = [(k[0],k[1],k[ 2],V)  for k,v in dict_list.items ()]    #  sorted by statistics, sorted and saved to the database.     for insert in sorted (ip_list,key=lambda x:x[3],reverse=true): # If you want to extract only the first 10 lines, you can [: 10].         print insert   #测试用的, can not. The whole function is to output one line and then save to the database.         sql =  ' insert loginfo values  ("%s", "% S ","%s ","%s ") '  % insert        print sql         cur.execute (SQL)

Create the corresponding table (based on the previous article)

Python parsing nginx log ip (source)

Use intest;show tables; CREATE TABLE ' loginfo ' (' IP ' text default null, ' URL ' longtext default null, ' status ' varchar ($) default NULL, ' Cou Nt_num ' varchar ($) DEFAULT NULL);

The results seen in the database after the script execution are as follows:

Mysql> select * from loginfo where count_num >3 limit 5  ;+----------------+-----------------------+--------+-----------+| ip              | url                    | status | count_num |+- ---------------+-----------------------+--------+-----------+| 121.42.0.85    |  /                      | 200    | 39        | |  121.42.0.37    | /                      | 403    | 34         | |  121.42.0.39    | /                      | 403    | 34         | |  222.95.248.220 | HTTP://www.baidu.com/ | 200    | 26         | |  23.251.49.10   | www.baidu.com:443     | 400     | 21        |+----------------+---------------- -------+--------+-----------+5 rows in set  (0.00 sec) mysql>

This article is from the "Struggle Bar" blog, please be sure to keep this source http://lvnian.blog.51cto.com/7155281/1845181

Python parsing nginx Log ip,url,status