Shell Script Analysis Nginx The most and time-consuming pages of log visits (slow query) _linux shell

When the server pressure is larger, running very laborious time. We often do Site page optimization, will go to find those pages are more frequent, and more time-consuming. Find the addresses that are high and time-consuming, and the related optimizations will get immediate results. Here is a piece of shell script that I use frequently when I do optimization. This can be counted as a Web page slowpage slow access page, like the MySQL slowquery.

Here is my: nginx preparation

Copy Code code as follows:

Log_format Main ' $remote _addr-$remote _user [$time _local] $request '
"$status" $body _bytes_sent "$http _referer"
' $http _user_agent ' $http _x_forwarded_for ' $request _time ';

Access_log/var/log/nginx/access.log main buffer=32k;

From the above configuration, you can see: IP in the first column, the page time consuming is in the last column, separated by a space in the middle. So in awk, you can use: $
$NF read to the current value. Where NF is a constant that represents the entire number of columns.

The following is the shell file for parsing code, which can be saved as a slow.sh

Copy Code code as follows:

#!/bin/sh

Export Path=/usr/bin:/bin:/usr/local/bin:/usr/x11r6/bin;
Export LANG=ZH_CN. GB2312;

function usage ()
{
echo "$ filelog Options";
Exit 1;
}

function Slowlog ()
{
#set-X;
field=$2;
files=$1;
end=2;
Msg= "";

[[$ = 1 ']] && field=1&&end=2&&msg= "Total visit count";
[[$ = 2 ']] && field=3&&end=4&&msg= "Average access time statistics";

ECHO-E "\r\n\r\n";
Echo-n "$msg";
Seq-s ' # ' 30 | Sed-e ' s/[0-9]*//g ';

awk ' {split ($7,BBB, "?"); arr[bbb[1]]=arr[bbb[1]]+ $NF; arr2[bbb[1]]=arr2[bbb[1]]+1; } end{for (i in arr) {print I: "Arr2[i]": "Arr[i]": "Arr[i]/arr2[i]}} ' $ | SORT-T: + $field-$end-rn |grep "pages" |head-30 | Sed ' s/:/\t/g '
}

[[$# < 2]] && usage;

Slowlog $ $;

Only need to execute: slow.sh log file 1 or 2
1:30 access to the most mundane pages
2:30 access to the most time-consuming pages

The results of the implementation are as follows:

chmod +x./slow.sh

chmod +x slow.sh
./slow.sh/var/log/nginx/
./slow.sh/var/log/nginx/access.log 2

Average Access time statistic #############################
/pages/####### #1. PHP 4 120.456 30.114
/pages/####### #2. PHP 1 16.161 16.161
/pages/####### #3. PHP 212 1122.49 5.29475
/pages/####### #4. PHP 6 28.645 4.77417

..................

I hope the above script will be helpful to everyone.