Awk web log analysis (page execution time) (common applications 3)

Today, we mainly talk about analyzing logs through awk to quickly get the execution time. The performance and efficiency are greatly improved compared with the previous article.

Some time ago, I wrote an article about how to use shell scripts to analyze the most frequently accessed and time-consuming pages (slow queries) of nginx logs, which mentioned the importance of time-consuming page analysis. Today, we mainly talk about analyzing logs through awk to quickly get the execution time. The performance and efficiency are greatly improved compared with the previous article!

I. web log file format

Copy codeThe code is as follows:
222.83.181.42--[09/Oct/2010: 04: 04: 03 + 0800] GET/pages/international/tejia. php HTTP/1.1 "200" 15708 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; Sicent; WoShiHoney. B ;. net clr 2.0.50727 ;. net clr 3.0.20.6.2152 ;. net clr 3.5.30729) "-" 0.037

If they are separated by spaces, the last field [0.037] is the page execution time, and the second field is the page access address.

II. execute code

Copy codeThe code is as follows:
Awk 'In in {
Print "Enter log file :";
Getline logs;
# Logs = "/var/log/nginx/access. log-20101008 ";
OFMT = "%. 3f ";

While (getline <logs)
{
Split ($7, atmp ,"? ");
AListNum [atmp [1] + = 1;
AListTime [atmp [1] + = $ NF;
Ilen ++;
}
Close (logs );
Print "\ r \ ntotal:", ilen, "\ r \ n ======================================== ===\ r \ n ";
For (k in aListNum)
{
Print k, aListNum [k], aListTime [k]/aListNum [k] | "sort-r-n-k3 ";
}

}'

Result:



Performance:



422780 logs. the statistical completion speed is about 5 seconds.