Fast and efficient analysis of IBM HTTP server logs using the awk command

IBM HTTP Server is a multifunctional Web server developed on top of the http://www.aliyun.com/zixun/aggregation/14417.html ">apache Web server." IBM HTTP Server extends Apache functionality and introduces features that Apache does not. For example, you can use the IBM Fast Response Cache accelerator and more easily integrate with other IBM WebSphere products. To understand the health of IBM HTTP server, a quick and efficient analysis of IBM HTTP server logs is required. Although there are a number of commercial, or open source software to provide a graphical user interface, to assist the analysis of server access logs, but most of them need to be installed and configured in advance on the server side. With the awk command, you can capture the required information more quickly from the log in many special situations. This article starts with the IBM HTTP Sever access log format, and then introduces the language base of AWK, and finally enumerates some typical usage scenarios.

IBM HTTP Server Access log format

Define the format of the log

We can use the predefined classic format or customize the format of the access log in IBM HTTP Server configuration files. If there are no special instructions below, assume that the log uses the classic format named combined.

Logformat "%h%l%u%t \%r\"%>s%b \ "%{referer}i\" \ "%{user-agent}i\" "combined

The following is a brief description of each domain:

%h = The client IP address that originated the request. The IP address recorded here is not necessarily the IP address of the real user client, it may be the public network mapping address or proxy server address of the private network client.

%l = Client's RFC 1413 identity (reference) and only clients that implement the RFC 1413 specification can provide this information. %u = Access User's ID%t = time%r = Request from client%>s = Server returns the client's status code%b = The byte size returned to the client, but does not include the size of the response header%{referer}i = reference page%{user-agent}i = browser's Type

The following three behavior sample logs:

202.189.63.115--[31/aug/2008:15:42:31 +0800] "get/http/1.1" 1365 "" mozilla/5.0 (Windows NT 6.1; WOW64; rv:15.0) gecko/20100101 firefox/15.0.1 "

Set up scroll logs

Because the WEB server can be a huge number of visits a day, we need to write the access log separately in different log files, which prevents a single file from being too large to be opened using the editor. For example, we can define a log file per 5 M in the configuration file.

Linux Server:

Transferlog "|/opt/ibm/httpserver/bin/rotatelogs/opt/ibm/httpserver/logs/access_log 5M"

Windows Server:

Customlog "| C:/ibm/httpserver/bin/rotatelogs.exe C:/ibm/httpserver/logs/access%y_%m_%d_%h_%m_%s.log 5M "combined