Linux Apache log analysis and Status View [go]

Suppose the Apache log format is:

118.78.199.98–-[09/jan/2010:00:59:59 +0800] "Get/public/css/index.css http/1.1″304–" http://www.a.cn/common/ index.php "" mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; GTB6.3) "

Issue 1: Find the 10 most visited IPs in Apachelog.

awk ' {print '} ' apache_log |sort |uniq-c|sort-nr|head-n 10

awk first grabs the IP in each log, such as the log format has been customized, you can define the delimiter and the print specified column.

Sort for the first order, so that the same records are arranged together;

Upiq-c merges duplicate rows and records the number of repetitions.

Head for the top 10 screening;

The SORT-NR is sorted by number in flashbacks.

The commands I refer to are:

Show 10 most common commands

Sed-e "s/| N/g "~/.bash_history | Cut-d '-F 1 | Sort | uniq-c | Sort-nr | Head

Issue 2: Find the most visited number of minutes in the Apache log.

awk ' {print $4} ' Access_log |cut-c 14-18|sort|uniq-c|sort-nr|head

The fourth column where awk is separated by a space is [09/jan/2010:00:59:59;

Cut-c extract 14 to 18 characters

The rest of the content is similar to question 1.

Issue 3: Find the most visited pages in the Apache log:

awk ' {print $11} ' Apache_log |sed ' s/^.*cn/(. */)/'//1/g ' |sort |uniq-c|sort-rn|head

Similar to questions 1 and 2, the only special is to replace "http://www.a.cn/common/index.php" in parentheses with the replacement function of sed: "http://www.a.cn (/common/index.php)"

Question 4: In the Apache log, find out the number of times (in minutes) of the most visited (heaviest load), and then look at these times which IP access is the most?

1. View Apache process:

PS aux | grep httpd | Grep-v grep | Wc-l

2, view the TCP connection for port 80:

Netstat-tan | grep "established" | grep ": 80" | Wc-l

3, through the log to view the number of IP connections today, filtering duplicates:

Cat Access_log | grep "19/may/2011" | awk ' {print $} ' | Sort | uniq-c | Sort-nr

4, what is the highest IP connection IP in the day (originally a spider):

Cat Access_log | grep "19/may/2011:00" | grep "61.135.166.230" | awk ' {print $8} ' | Sort | uniq-c | Sort-nr | Head-n 10

5, the first page of the day to access the top 10 URL:

Cat Access_log | grep "19/may/2010:00" | awk ' {print $8} ' | Sort | uniq-c | Sort-nr | Head-n 10

6, Sniff with tcpdump 80 port to see who's highest

Tcpdump-i ETH0-TNN DST Port 80-c 1000 | Awk-f "." ' {print $ '. $ "." $ "." $4} ' | Sort | uniq-c | Sort-nr

Then check the log to see what the IP is doing:

Cat Access_log | grep 220.181.38.183| awk ' {print $ '/t ' $8} ' | Sort | uniq-c | Sort-nr | Less

7. View the number of IP connections for a time period:

grep "2006:0[7-8]" Www20110519.log | awk ' {print $} ' | Sort | uniq-c| Sort-nr | Wc-l

8, the maximum number of 20 IP addresses in the current Web server that are joined:

Netstat-ntu |awk ' {print $} ' |sort | uniq-c| Sort-n-R | Head-n 20

9, view the top 10 most visited IPs in the log

Cat Access_log |cut-d '-F 1 |sort |uniq-c | Sort-nr | awk ' {print $} ' | Head-n |less

10, see more than 100 IPs in the log

Cat Access_log |cut-d '-F 1 |sort |uniq-c | awk ' {if (>) print $ |sort-nr} ' |less

11. View the most recently accessed files

Cat Access_log |tail-10000|awk ' {print $7} ' |sort|uniq-c|sort-nr|less

12. View pages with more than 100 visits in the log

Cat Access_log | Cut-d '-F 7 | Sort |uniq-c | awk ' {if (>) print $} ' | Less

13, list files with transmission time exceeding 30 seconds

Cat Access_log|awk ' ($NF >) {print $7} ' |sort-n|uniq-c|sort-nr|head-20

14, List the most time-consuming pages (more than 60 seconds) and the number of corresponding page occurrences

Cat Access_log |awk ' ($NF > && $7~//.php/) {print $7} ' |sort-n|uniq-c|sort-nr|head-100