Essential solutions and Log Analysis for CentOS website server maintenance

Log Analysis command to analyze the URL (504 error, errors greater than 500 can be written as 5 ..) the following command is a list of URLs with 504 errors. Awk & #39; $10 ~ /504/{print $8} & #39; xxx_com_access.log-$ {date} | awk-F & quot; & amp; & quot; & #39; {print $1, $2} & #39; | sort | uni log analysis command

Analyze the URL (504 error, more than 500 error can be directly written as 5 ..)
The following command lists URLs with 504 errors.

awk '$10 ~ /504/ { print $8}' xxx_com_access.log-${date} | awk -F "&" '{print $1,$2}' | sort  | uniq -c | sort -rn | head -10 

Analysis Server Response Time (greater than 100 ms)
The following command displays a list of servers whose response time is greater than Ms.

awk '{if($NF>0.10) print $(NF-1)}' xxx_com_access.log | uniq -c | sort -n 

 

Find the minutes with the most visits:

awk '{print $5}' xxxx_com_access.log |cut -c 14-18|sort|uniq -c|sort -nr|head

Top 15 most accessed by IP addresses in logs

awk '{ip[$2]++} ; END{ for (i in ip) print ip[i],i }' xxx_access.log-20130922  | sort -n | tail -n 15

Suspicious ip Analysis

Statistics on the mac address of the client corresponding to the IP Address: awk '$2 ~ /219.148.23.36/{print $2, $8} 'xxxx_access.log-20130922 | grep mac = | awk-F "="' {print $(NF-1 )} '| grep-v mac | cut-c 1-17 | sort | uniq-c | user-agent statistics corresponding to the sort-nip address: awk' $2 ~ /111.30.132.190/{print $2, $5, $15, $16, $17, $18, $19, $20} 'xxx_com_access.log-20130922 | more

 

There are also:

Ip address with the highest traffic: tcpdump-I em1-tnn dst port 80-c 100000 | awk-F ". "'{print $1 ". "$2 ". "$3 ". "$4} '| sort | uniq-c | sort-nr | head-n 15 find out what this ip address is doing: cd/data/logs/xxx; tail-n 10000000 xxx_com_access.log | grep 123.150.205.205 | awk '{print $2 "\ t" $8}' | sort | uniq-c | sort-nr