Shell script statistics in Linux page traffic in Apache log

Source: Internet
Author: User
Tags apache log

Requirements: There are two pages in the program for a purpose, and now you need to count the number of visits per minute between the two pages, including total hits, number of successes (status code 200), and number of failures. Then write to MySQL.

MySQL fields: IDs (from growth), time (actual statistics), year, month, day, hour, visit (Total hits), success, fail, target (destination)

#!/bin/bash
#Desc: Analysis of Apache log for witch pages with one minutes
#install BC packages with yum install-y BC if not found BC command

Logdir=/var/log/httpd/80/access
logfile= $logdir/$ (date +%y%m%d) _access_log

month=$ (Date|awk ' {print$2} ')
day=$ (Date|awk ' {print$3} ')
year=$ (Date|awk ' {print$6} ')
hour=$ (Date|awk ' {print$4} ' |cut-d:-F1)
min=$ (Date|awk ' {print$4} ' |cut-d:-F2)

# Get current system times to/tmp
echo $month >/tmp/month.txt
echo $day >/tmp/day.txt
echo $year >/tmp/year.txt
echo $hour >/tmp/hour.txt
echo $min >/tmp/min.txt

# Compute Run minute
If ["$min" = = "00"]; Then
run_minute=59
Else
If ["$ (awk-f ' {print$ (NF-1)} '/tmp/min.txt)" = = "0"]; Then
echo $ (awk-f ' {print$nf} '/tmp/min.txt) >/tmp/min2.txt
echo $ (cat/tmp/min2.txt) -1|BC >/tmp/run_minute.txt

If ["cat/tmp/run_minute.txt" = = "0"]; Then
run_minute=00
Else
run_minute=0$ (Cat/tmp/run_minute.txt)
Fi

Else

echo $ (cat/tmp/min.txt) -1|BC >/tmp/run_minute.txt
If ["$ (cat/tmp/run_minute.txt)" = = "9"]; Then
run_minute=09
Else
run_minute=$ (Cat/tmp/run_minute.txt)
Fi
Fi

Fi

# COMPUTE Run Hour
If ["$hour" = = "00"]; Then
If ["$min" = = "00"]; Then
Run_hour=23
Else
Run_hour= $hour
Fi
Else
If ["$min" = = "00"]; Then
If ["$ (awk-f ' {print$ (NF-1)} '/tmp/hour.txt)" = = "0"]; Then
echo $ (awk-f ' {print$nf} '/tmp/hour.txt) >/tmp/hour2.txt
echo $ (cat/tmp/hour2.txt) -1|BC >/tmp/run_hour.txt
If ["$ (cat/tmp/run_hour.txt)" = = "0"]; Then
run_hour=00
Else
run_hour=0$ (Cat/tmp/run_hour.txt)
Fi
Else
echo $ (cat/tmp/hour.txt) -1|BC >/tmp/run_hour.txt
run_hour=$ (Cat/tmp/run_hour.txt)
Fi
Else
Run_hour= $hour
Fi
Fi

# Get run log file
If ["$hour" = = "00"]; Then
If ["$min" = = "00"]; Then
run_log= $logdir/$ (date-d last-day +%y%m%d_access_log)
Else
run_log= $logfile
Fi
Else
run_log= $logfile
Fi

# BEGIN Log Analysis
p1=cnbpush.php
p2=jlpush.php

If ["$hour" = = "00"]; Then
If ["$min" = = "00"]; Then
insert_time=$ (date-d '-1 hours ' +%y%m%d |cut-c 3-8) $RUN _hour$run_minute
Else
insert_time=$ (date +%y%m%d |cut-c 3-8) $RUN _hour$run_minute
Fi
Else
insert_time=$ (date +%y%m%d |cut-c 3-8) $RUN _hour$run_minute

Fi

echo $INSERT _time | Cut-c 1-2 >/tmp/iy
echo $INSERT _time | Cut-c 3-4 >/tmp/im
echo $INSERT _time | Cut-c 5-6 >/tmp/id
echo $INSERT _time | Cut-c 7-8 >/tmp/ih
insert_year=20$ (Cat/tmp/iy)
insert_month=$ (CAT/TMP/IM)
insert_day=$ (CAT/TMP/ID)
insert_hour=$ (CAT/TMP/IH)

cn_total_access=$ (cat $RUN _log | grep "$p 1" | awk-f ': ' {print$2$3} ' | grep "$RUN _hour$run_minute" | wc-l)
cn_succ_access=$ (cat $RUN _log | grep "$p 1" | grep "| awk-f ': ' {print$2$3} ' | grep" $RUN _hour$run_minute "| wc-l)
cn_fail_access=$ (cat $RUN _log | grep "$p 1" | grep-v "| awk-f ': ' {print$2$3} ' | grep" $RUN _hour$run_minute "| wc-l )

jl_total_access=$ (cat $RUN _log | grep "$p 2" | awk-f ': ' {print$2$3} ' | grep "$RUN _hour$run_minute" | wc-l)
jl_succ_access=$ (cat $RUN _log | grep "$p 2" | grep "| awk-f ': ' {print$2$3} ' | grep" $RUN _hour$run_minute "| wc-l)
jl_fail_access=$ (cat $RUN _log | grep "$p 2" | grep-v "| awk-f ': ' {print$2$3} ' | grep" $RUN _hour$run_minute "| wc-l )

Db_name=shell_log
Db_user=root
Db_pass=root
Db_tab=log

Mysql-u$db_user-p$db_pass $DB _name-e "INSERT into $DB _tab (time,year,month,day,hour,visit,success,fail,target) VALUES (' $INSERT _time ', ' $INSERT _year ', ' $INSERT _month ', ' $INSERT _day ', ' $INSERT _hour ', ' $CN _total_access ', ' $CN _succ _access ', ' $CN _fail_access ', ' Cnbpush ');
Mysql-u$db_user-p$db_pass $DB _name-e "INSERT into $DB _tab (time,year,month,day,hour,visit,success,fail,target) VALUES (' $INSERT _time ', ' $INSERT _year ', ' $INSERT _month ', ' $INSERT _day ', ' $INSERT _hour ', ' $JL _total_access ', ' $JL _succ _access ', ' $JL _fail_access ', ' Jlpush ');

Exit

Put it on a task schedule, one minute to execute.

Related Article

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: info-contact@alibabacloud.com and provide relevant evidence. A staff member will contact you within 5 working days.

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.