Command set for analyzing apache logs using grep in linux

Linux grep command set for analyzing apache logs in linux. a good article is worth noting. with these commands, apache log analysis is a major part of the history. Www.2cto.com instance: month. january II... linux grep command set for analyzing apache logs in linux. a good article is worth noting. with these commands, apache log analysis is a major part of the history. Www.2cto.com instance: month. january Feb. february Mar. march April Apr. april May. may June. june July. july Aug. aguest Sept. september October Oct. october Nov. november Dec December. december www.2cto.com log analysis and collation Grep log arrangement 1. analyze the top 20 URLs on the access page in and sort cat access. log | grep '04/May/2012 '| awk' {print $11} '| sort | uniq-c | sort-nr | head-20 query the URL address of the Accessed page contains the IP address cat access_l of www.abc.com. Og | awk '($11 ~ /\ Www.abc.com/) {print $1} '| sort | uniq-c | sort-nr 2. get the Top 10 IP addresses and query cat linewow-access.log by time | awk '{print $1}' | sort | uniq-c | sort-nr | head-10 1. obtain the first 10 IP addresses of cat access. log | awk '{print $1}' | sort | uniq-c | sort-nr | head-10 cat access. log | awk '{counts [$ (11)] + = 1}; END {for (url in counts) print counts [url], url}' 2. take the first 20 files or pages with the most visits and count all access IP addresses. log | awk '{print $11}' | sort | uniq-c | sort-nr | Head-20 awk '{print $1}' access. log | sort-n-r | uniq-c | wc-l cat wangsu. log | egrep '06/Sep/| 06/Sep/'| awk' {print $1} '| sort | uniq-c | sort-nr | head-10 Query 3. list the largest number of exe files transmitted (commonly used when analyzing download sites) cat access. log | awk '($7 ~ /\. Exe/) {print $10 "" $1 "" $4 "" $7} '| sort-nr | head-20 4. lists the exe files with an output greater than 200000 bytes (about KB) and the number of occurrences of the corresponding files cat access. log | awk '($10> 200000 & $7 ~ /\. Exe/) {print $7} '| sort-n | uniq-c | sort-nr | head-100 5. if the last column of the log records the page file transfer time, the most time-consuming page cat access is listed on the client. log | awk '($7 ~ /\. Php/) {print $ NF "" $1 "" $4 "" $7} '| sort-nr | head-100 6. list the most time-consuming pages (more than 60 seconds) and the corresponding page occurrence times cat access. log | awk '($ NF> 60 & $7 ~ /\. Php/) {print $7} '| sort-n | uniq-c | sort-nr | head-100 7. lists the cat access files that have been transferred for more than 30 seconds. log | awk '($ NF> 30) {print $7}' | sort-n | uniq-c | sort-nr | head-20 8. count website traffic (G) cat access. log | awk '{sum + = $10} END {print sum/1024/1024/1024}' 9. count 404 of connected awk' ($9 ~ /404/) 'Access. log | awk '{print $9, $7}' | sort 10. count http status. cat access. log | awk '{counts [$ (9)] + = 1}; END {for (code in counts) print code, counts [code]} 'cat access. log | awk '{print $9}' | sort | uniq-c | sort-rn 11. concurrency per second: awk '{if ($9 ~ /200 | 30 | 404/) COUNT [$4] ++} END {for (a in COUNT) print, COUNT [a]} '| sort-k 2-nr | head-n10 12. bandwidth statistics cat apache. log | awk '{if ($7 ~ /GET/) count ++} END {print "client_request =" count} 'cat apache. log | awk '{BYTE + = $11} END {print "client_kbyte_out =" BYTE/1024 "KB"}' find the 10 most frequently accessed IP addresses in a day cat/tmp/access. log | grep "20/Mar/2011" | awk '{print $3}' | sort | uniq-c | sort-nr | the ip addresses with the highest number of connections on the day of the head are all working. what: cat access. log | grep "10.0.21.17" | awk '{print $8}' | sort | uniq-c | sort-nr | head-n 10 find the most frequently accessed minutes of awk '{ print $1} 'Access. log | grep "20/Mar/2011" | cut-c 14-18 | sort | uniq-c | sort-nr | head attachment: view tcp connection status netstat-nat | awk '{print $6}' | sort | uniq-c | sort-rn netstat-n | awk '/^ tcp/{++ S [$ NF]}; END {for (a in S) print a, S [a]} 'netstat-n | awk'/^ tcp/{++ state [$ NF]}; END {for (key in state) print key, "\ t ", state [key]} 'netstat-n | awk'/^ tcp/{++ arr [$ NF]}; END {for (k in arr) print k, "\ t ", arr [k]} 'netstat-n | awk'/^ tcp/{print $ NF} '| sort | uniq-c | sort-rn netstat-ant | awk' {print $ NF} '| grep-V' [a-z]' | sort | uniq-c netstat-ant | awk '/ip: 80/{split ($5, ip, ":"); ++ S [ip [1]} END {for (a in S) print S [a], a} '| sort-n netstat-ant | awk'/: 80/{split ($5, ip ,":"); ++ S [ip [1]} END {for (a in S) print S [a], a} '| sort-rn | head-n 10 awk' BEGIN {printf ("http_code \ tcount_num \ n ")} {COUNT [$10] ++} END {for (a in COUNT) printf a "\ t" COUNT [a] "\ n"} '2. for more than 20 requests, see netstat-anlp | grep 80 | grep tcp | awk '{print $5}' | awk-F: '{print $1}' | sort | uniq-c | sort-nr | head-n20 netstat-ant | awk '/: 80/{split ($5, ip, ":"); ++ A [ip [1]} END {for (I in A) print A [I], i} '| sort-rn | head-n20 3. use tcpdump to sniff access to port 80 to see who has the highest tcpdump-I eth0-tnn dst port 80-c 1000 | awk-F ". "'{print $1 ". "$2 ". "$3 ". "$4} '| sort | uniq-c | sort-nr | head-20 4. find more time_wait connections to netstat-n | grep TIME_WAIT | awk '{print $5}' | sort | uniq-c | sort-rn | head-n20 5. find more SYN connections. netstat-an | grep SYN | awk '{print $5}' | awk-F: '{print $1}' | sort | uniq-c | sort-nr | more 6. process netstat-ntlp | grep 80 | awk '{print $7}' | cut-d/-f1 based on the port column