Linux File Log filtering operations

Source: Internet
Author: User
Tags log log sorts

Statistics view files and filter logs

1. Count the number of independent IPs in the *.log log file:
' {print $} ' Test.log | Sort | Uniq | Wc-l
2. #查询访问最多的前10个ip
' {print $} ' /access.log  | sort | uniq-c | sort-nr | head-Ten
3. #查看某段时间的
" 2017:0[3-6] "
4, the most visited IP
Netstat-ntu | Tail-n +3'{print$15
Tail-n +3: Remove the first two lines. awk ' {print $} ': Low 5 fields (5th column) Cut-d:-F 1: Take IP section. Sort: Sorts the IP parts. UNIQ-C: Prints the number of occurrences of each repeating row. (and remove duplicate rows) Sort-n-r: In reverse order in which repeated rows appear. Head-n 5: Take the top 5-bit IP5, shell statistics a day Access.log logs per hour per IP access:
awk-vfs="[:]"'{gsub ("-.*", "" ", $ $), num[$2" "$1]++}end{for (i in num) print I, Num[i]}' test.log
6. Filter keywords

' 01/sep/2017:16:06:47 ' logs/access.log /opt/mongodb/log/mongodb.log. - -10t05-- ten "Dec"

Sedsed-n '/dec 10/p '/opt/mongod/log/mongod.log awkawk '/dec 10/{print $} '/opt/mongod/log/mongod.log6, specific point-in-time logs;
sedsed-N'/nov 16:24:17/p'/var/log/Secureawkawk'/nov 16:24:17/{print $}'/var/log/Securetail-NTenTest.log Log at the end of the last 10 rows of the query log; tail-N +TenTest.log queries all logs after 10 rows; head-NTenTest.log The first 10 rows of logs in the query log file; Head-N-Tentest.log query log file except for all other logs in the last 10 lines;-N Test.log |tail-n + the|head-n -Tail-N +92 indicates the log head after querying 92 rows-N -The first 20 records are found in the previous query results.
7. Find logs on specified time end
' /2014-12-17 16:17:20/,/2014-12-17 16:17:36/p ' ' 2014-12-1716:17:20'    
8, use >xxx.txt to save it to a file, you can pull down this file analysis. such as:
"  Awesome "  >test.log

Netstat-ntu | Tail-n +3 | awk ' {print $} ' | Cut-d:-F 1 | Sort | uniq-c| Sort-n-R | Head-n 5 tail-n +3: Remove the first two lines. awk ' {print $} ': Low 5 fields (5th column) Cut-d:-F 1: Take IP section. Sort: Sorts the IP parts. UNIQ-C: Prints the number of occurrences of each repeating row. (and remove duplicate rows) Sort-n-r: In reverse order in which repeated rows appear. Head-n 5: Take the top 5-bit ip  5, the shell statistics a day Access.log log per hour per IP access:  awk-vfs= "[:]" ' {gsub ("-.*", "", $ $); Num[$2 "" $1]++}end{for (i in num) print I,num[i]} ' logs/access.log  6, grep filter keyword  grep ' 01/sep/2017:16:06:47 ' logs /access.log cat/opt/mongodb/log/mongodb.log.2016-12-10t05-36-42 |grep "Dec"  sedsed-n '/Dec 10/p '/opt/ Mongod/log/mongod.log awkawk '/dec 10/{print $} '/opt/mongod/log/mongod.log 6, specific point-in-time log; Sedsed-n '/Nov 11 16 : 24:17/p '/var/log/secure awkawk '/nov one 16:24:17/{print $} '/var/log/secure tail-n test.log query log trailing last 10 lines Tail-n +10 test.log Query all logs after 10 rows, head-n test.log Query the header 10 rows in the log file, head-n -10 test.log query log file except for all other logs in the last 10 rows;  c At-n Test.log |tail-n +92|head-n 20Tail-n +92 indicates that 92 rows after the query of the log head-n 20 means that the previous query results in the first 20 records  7, find the log of the specified time Sed-n '/2014-12-17 16:17:20/,/2014-12-17 16:17:36/p ' test.log grep ' 2014-12-17 16:17:20 ' test.log 8, using >xxx.txt to save it to a file, you can pull down this file analysis. For example: Cat-n Test.log |grep "Terrain" >xxx.txt

Linux file log filtering operations

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: info-contact@alibabacloud.com and provide relevant evidence. A staff member will contact you within 5 working days.

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.