awk parsing Web logs (page execution time)

Shell Script Analysis Nginx log most visited and the most time-consuming page (slow query) when the server pressure is relatively large, run up very laborious time. We often do Site page optimization, will go to find those pages to visit more times, and more time-consuming. Find those with high access times, and more time-consuming addresses, on-line optimization, will achieve immediate results. Here are some of the shell scripts that I used frequently when I was doing optimization. This also can be counted, the statistical web page slowpage Slow access page, like MySQL slowquery. Here's my: Nginx compounding

Log_format Main ' $remote _addr-$remote _user [$time _local] $request '
"$status" $body _bytes_sent "$http _referer"
' "$http _user_agent" "$http _x_forwarded_for" $request _time ';
Access_log/var/log/nginx/access.log main buffer=32k;

 From the above configuration, you can see: IP in the first column, the page is time-consuming in the last column, the middle is separated by a space. So in awk, you can use: $1$NF to read to the current value, respectively. Where NF is a constant, representing the entire number of columns. Quickly get execution time by using awk to parse logs. Web log file format

222.83.181.42--[09/oct/2010:04:04:03 +0800] get/pages/international/tejia.php http/1.1 "" "15708"-"" mozilla/4.0 (c ompatible; MSIE 6.0; Windows NT 5.1; SV1; Sicent; woshihoney.b;. NET CLR 2.0.50727;. NET CLR 3.0.4506.2152;. NET CLR 3.5.30729) ""-"0.037
Separated by a space, the last field [0.037] is the page execution time, and the 7th field is the page access address.

 Ii. Code of execution

awk ' begin{
Print "Enter log file:";
Getline logs;
#logs = "/var/log/nginx/access.log-20101008";
Ofmt= "%.3f";
while (Getline < logs)
{
Split ($7,atmp, "?");
Alistnum[atmp[1]]+=1;
alisttime[atmp[1]]+= $NF;
ilen++;
}
Close (logs);
Print "\r\ntotal:", Ilen, "\r\n======================================\r\n";
For (k in Alistnum)
{
Print K,alistnum[k],alisttime[k]/alistnum[k] | "Sort-r-n-k3";
}
}‘

Results: Performance: 422,780 logs, statistical completion speed is: about 5 seconds.

awk parsing Web logs (page execution time)