Script sharing: According to the Access log Statistics site daily PV, total bandwidth, IP statistics, URL statistics, etc.

In peacetime operation and maintenance work, our OPS personnel need to know the total number of daily visits, total bandwidth, IP statistics and URL statistics of their website.

Although the website has already done CDN acceleration in the service provider, so the website traffic pressure is in the front CDN layer

Like daily PV, bandwidth, IP statistics and other data can also be found in their backstage.

------------------------------------------------------------------------------
Here, or share a very early use of the shell statistics script, can be combined with crontab scheduled tasks, daily to your mailbox to send a statistical report ~
The script counts:
1, the total number of visits
2. Total Bandwidth
3. Number of independent visitors
4. Access IP Statistics
5. Access URL Statistics
6. SOURCE Statistics
7, 404 statistics
8, search engine access statistics (Google, Baidu)
9, search engine source statistics (Google, Baidu)

[email protected] ~]# cat tongji.sh
#!/bin/bash
Log_path=/data/logs/nginx/www.huanqiu.com/access.log
Domain= "huanqiu.com"
Email= "[email protected]"
Maketime= ' Date +%y-%m-%d ' "%H": "%m"
Logdate= ' date-d "Yesterday" +%y-%m-%d '
Total_visit= ' Wc-l ${log_path} | awk ' {print '} '
Total_bandwidth= ' Awk-v total=0 ' {total+=$10}end{print total/1024/1024} ' ${log_path} '
Total_unique= ' awk ' {ip[$1]++}end{print asort (IP)} ' ${log_path} '
ip_pv= ' awk ' {ip[$1]++}end{for (k in IP) {print ip[k],k}} ' ${log_path} | Sort-rn | Head-20 '
Url_num= ' awk ' {url[$7]++}end{for (k in URL) {print url[k],k}} ' ${log_path} | Sort-rn | Head-20 '
Referer= ' awk-v domain= $domain ' $11!~/http:\/\/[^/]* ' "$domain" '/{url[$11]++}end{for (k in URL) {print url[k],k}} ' ${log _path} | Sort-rn | Head-20 '
Notfound= ' awk ' $9 = = 404 {url[$7]++}end{for (k in URL) {print url[k],k}} ' ${log_path} | Sort-rn | Head-20 '
Spider= ' awk-f ' "" $6 ~/baiduspider/{spider["Baiduspider"]++} $6 ~/googlebot/{spider["Googlebot"]++}end{for (k in SPID ER) {print k,spider[k]}} ' ${log_path} '
Search= ' awk-f ' "" $4 ~/http:\/\/www\.baidu\.com/{search["Baidu_search"]++} $4 ~/http:\/\/www\.google\.com/{search[ "Google_search"]++}end{for (k in search) {print k,search[k]}} ' ${log_path} '
Echo-e "profile \ n Report generated time: ${maketime}\n Total Visits: ${total_visit}\n Total Bandwidth: ${total_bandwidth}m\n Independent Visitors: ${total_unique}\n\n Access IP Statistics \n$ {ip_pv}\n\n Access URL Statistics \n${url_num}\n\n source page Statistics \n${referer}\n\n404 statistics \n${notfound}\n\n spider statistics \n${spider}\n\n search engine source Statistics \n${ Search} "| Mail-s "$domain $logdate Log Statistics" ${email}

The above script can be applied to other website statistics
You only need to modify the three variables in the script above:
Log_path
Domain
Email
By adding this script to the scheduled task, you can receive the statistical data every day.

Execute the above script and go to the [email protected] Mailbox to view the statistics report:

[Email protected] ~]# sh tongji.sh
[Email protected] ~]#

[[Email protected] ~] #crontab-E
#每天凌晨按时统计一次
* * * * */bin/bash-x/root/tongji.sh >/dev/null 2>&1

Script sharing: According to the Access log Statistics site daily PV, total bandwidth, IP statistics, URL statistics, etc.