shell （awk）數組使用案例

標籤：awk 數組

1.取ifconfig eth0的IP地址

[[email protected] ~]$ ifconfig eth0

eth0      Link encap:Ethernet  HWaddr 00:0C:29:68:47:71  

          inet addr:192.168.48.144  Bcast:192.168.48.255  Mask:255.255.255.0

          inet6 addr: fe80::20c:29ff:fe68:4771/64 Scope:Link

          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1

          RX packets:48161 errors:0 dropped:0 overruns:0 frame:0

          TX packets:25022 errors:0 dropped:0 overruns:0 carrier:0

          collisions:0 txqueuelen:1000 

          RX bytes:36048837 (34.3 MiB)  TX bytes:2202106 (2.1 MiB)

[[email protected] ~]$ ifconfig eth0|awk -F ‘[ :]+‘ ‘NR==2{print NR,$4}‘

2 192.168.48.144

2.統計apache日誌單IP訪問請求數排名

[[email protected] ~]$ cat 2.txt 

10.0.0.3 -- [21/Mar/2015-07:50:17+0800]*GET/HTTP/1.1*200 19 *-*

10.0.0.3 -- [21/Mar/2015-07:50:17+0800]*GET/HTTP/1.1*200 19 *-*

10.0.0.5 -- [21/Mar/2015-07:50:17+0800]*GET/HTTP/1.1*200 19 *-*

10.0.0.3 -- [21/Mar/2015-07:50:17+0800]*GET/HTTP/1.1*200 19 *-*

10.0.0.3 -- [21/Mar/2015-07:50:17+0800]*GET/HTTP/1.1*200 19 *-*

10.0.0.6 -- [21/Mar/2015-07:50:17+0800]*GET/HTTP/1.1*200 19 *-*

10.0.0.3 -- [21/Mar/2015-07:50:17+0800]*GET/HTTP/1.1*200 19 *-*

10.0.0.3 -- [21/Mar/2015-07:50:17+0800]*GET/HTTP/1.1*200 19 *-*

[[email protected] ~]$ awk ‘{print $1}‘ 2.txt |sort|uniq -c

      6 10.0.0.3

      1 10.0.0.5

      1 10.0.0.6

[[email protected] ~]$ awk ‘{array[$1]++}END{for(k in array){print k,array[k];}}‘ 2.txt 

10.0.0.3 6

10.0.0.5 1

10.0.0.6 1

3.統計網域名稱訪問量

[[email protected] ~]$ cat 1.txt 

http://www.baidu.com/index.html

http://www.163.com/1.html

http://www.cnblogs.com/index.html

http://www.baidu.com/2.html

http://www.163.com/index.html

http://www.qq.com/index.html

http://www.baidu.com/3.html

http://www.163.com/2.html

http://www.baidu.com/2.html

[[email protected] ~]$ awk ‘{split($0,array,"/+");key=array[2];count[key]++}END{for(kk in count) {print kk,count[kk]}}‘ 1.txt

www.qq.com 1

www.cnblogs.com 1

www.baidu.com 4

www.163.com 3

[[email protected] ~]$ awk -F "[ /]+" ‘{array[$2]++}END{for(i in array){print i,array[i]}}‘ 1.txt 

www.qq.com 1

www.cnblogs.com 1

www.baidu.com 4

www.163.com 3

4.計算每個人的總工資和平均工資

[[email protected] ~]$ awk ‘{array[$2]+=$3;count[$2]++}END{for(key in count){print key,array[key]"k",array[key]/count[key]}}‘ 3.txt 

kuqi 20k 10

jeacen 14k 7

yingsui 27k 13.5

xiaofen 16k 8

wodi 23k 11.5

yideng 21k 10.5

5.對本地IP和遠程IP去重並統計重複數

[[email protected] ~]$ cat 4.txt 

Proto Recv-Q Send-Q Local Addree Foreign Addree              State

 

tcp            0         0 0.0.0.0:80         0.0.0.0:*     LISTEN

tcp            0         0 115.29.49.213:80 117.136.27.254:13779     SYN_RECV

tcp            0         0 115.29.49.213:80     113.97.117.157:1847     SYN_RECV

tcp            0         0 115.29.49.213:80     117.136.40.20:19594     SYN_RECV

tcp            0         0 115.29.49.213:80     117.136.40.20:19595     SYN_RECV

tcp            0         0 115.29.49.213:80     121.236.219.69:45363     SYN_RECV

tcp            0         0 0.0.0.0:21         0.0.0.0:*     LISTEN

tcp            0         0 0.0.0.0:22     0.0.0.0:*         LISTEN

unix         3 [] SYREAM CONNECTED 11183664 /TMP/MYSQL.SOCK

unix         3 [] SYREAM CONNECTED 11183646 /TMP/MYSQL.SOCK

unix         3 [] SYREAM CONNECTED 11183665 /TMP/MYSQL.SOCK

unix         3 [] SYREAM CONNECTED 11183668 /TMP/MYSQL.SOCK

unix         3 [] SYREAM CONNECTED 11183654 /TMP/MYSQL.SOCK

unix         3 [] SYREAM CONNECTED 11183655 /TMP/MYSQL.SOCK

unix         3 [] SYREAM CONNECTED 11183668 /TMP/MYSQL.SOCK

unix         3 [] SYREAM CONNECTED 11183676 /TMP/MYSQL.SOCK

unix         3 [] SYREAM CONNECTED 11183672 /TMP/MYSQL.SOCK

[[email protected] ~]$ awk -F ‘[ :]+‘ ‘/^tcp/{array[$4" "$6]++}END{for(key in array){print key,array[key]}}‘ 4.txt 

115.29.49.213 117.136.40.20 2

0.0.0.0 0.0.0.0 3

115.29.49.213 113.97.117.157 1

115.29.49.213 121.236.219.69 1

115.29.49.213 117.136.27.254 1

6.統計源IP連接埠及目的IP同時去重

[[email protected] ~]$ cat 5.txt 

Dec 2 01:17:42 10.0.0.0 2009 RV016 RGFW-OUT:ACCEPT (TCP 10.0.0.131:1227->210.192.121.172:80 on ixp7) [0,0]

Dec 2 01:17:42 10.0.0.0 2009 RV016 RGFW-OUT:ACCEPT (TCP 10.0.0.131:1227->210.192.121.172:80 on ixp7) [0,0]

Dec 2 01:17:42 10.0.0.0 2009 RV016 RGFW-OUT:ACCEPT (TCP 10.0.0.131:1227->210.192.121.172:80 on ixp7) [0,0]

Dec 2 01:17:42 10.0.0.0 2009 RV016 RGFW-OUT:ACCEPT (TCP 10.0.0.131:1227->210.192.121.172:80 on ixp7) [0,0]

Dec 2 01:17:42 10.0.0.0 2009 RV016 RGFW-OUT:ACCEPT (TCP 10.0.0.131:1227->210.192.121.172:80 on ixp7) [0,0]

Dec 2 01:17:42 10.0.0.0 2009 RV016 RGFW-OUT:ACCEPT (TCP 10.0.0.43:54963->203.81.19.92:80 on ppp6) [0,0]

Dec 2 01:17:42 10.0.0.0 2009 RV016 RGFW-OUT:ACCEPT (UDP 10.0.0.19:1441->121.14.96.233:80 on ppp0) [0,0]

Dec 2 01:17:42 10.0.0.0 2009 RV016 RGFW-OUT:ACCEPT (UDP 172.16.1.103:57318->211.147.6.3:80 on ppp2) [0,0]

Dec 2 01:17:42 10.0.0.0 2009 RV016 RGFW-OUT:ACCEPT (TCP 172.16.1.203:4372->61.135.163.86:80 on ixp7) [0,0]

Dec 2 01:17:42 10.0.0.0 2009 RV016 RGFW-OUT:ACCEPT (TCP 10.0.0.131:1227->210.192.121.172:80 on ixp7) [0,0]

[[email protected] ~]$ vim 5.sh

{

split($9,array,":|->")

sip=array[1]

sport=array[2]

mip=array[3]

if (!((sip,sport,mip) in tee)) {

tee[sip,sport,mip] = 1

}

}

END{

for (key in tee)

print key

}

[[email protected] ~]$ awk -f 5.sh 5.txt 

172.16.1.103 57318    211.147.6.3

10.0.0.19    1441      121.14.96.233

172.16.1.203 4372    61.135.163.86

10.0.0.43    54963    203.81.19.92

10.0.0.131   1227     210.192.121.172

最佳化效果如下

在5.sh開頭添加BEGIN { printf("%-16s %-6s %-16s\n","SIP","SPORT","MIP")}

[[email protected] ~]$ awk -f 5.sh 5.txt 

SIP              SPORT  MIP             

172.16.1.10357318211.147.6.3

10.0.0.191441121.14.96.233

172.16.1.203437261.135.163.86

10.0.0.4354963203.81.19.92

10.0.0.1311227210.192.121.172

shell （awk）數組使用案例