AWK及PYTHON求和及平均值方法

檔案如下

# cat cesc
a,1
a,2
b,3
b,4
c,2
d,5

需要擷取abcd出現的次數，逗號後面數位和及平均值。

With shell：

# grep -E ^a cesc |awk -F ',' '{sum+=$2} END {print "a, Count:" NR " Sum: " sum " Average: " sum/NR}'
a, Count:2 Sum: 3 Average: 1.5
# grep -E ^b cesc |awk -F ',' '{sum+=$2} END {print "b, Count:" NR " Sum: " sum " Average: " sum/NR}'
b, Count:2 Sum: 7 Average: 3.5
# grep -E ^c cesc |awk -F ',' '{sum+=$2} END {print "c, Count:" NR " Sum: " sum " Average: " sum/NR}'
c, Count:1 Sum: 2 Average: 2
# grep -E ^d cesc |awk -F ',' '{sum+=$2} END {print "d, Count:" NR " Sum: " sum " Average: " sum/NR}'
d, Count:1 Sum: 5 Average: 5

或者寫成一個for迴圈，這樣可移植性更好，另外，在awk中引用shell的變數有兩種辦法，一個是用雙引號和單引號包含變數，如：”‘var'”，還有就是使用awk的-v參數提前聲明，如：awk -v var=”$var”

# for i in `cat cesc |cut -d, -f1|sort|uniq`;do grep -E ^$i cesc |awk -F ',' '{sum+=$2} END {print "'$i'" " Count: " NR ", Sum: " sum ", Average: " sum/NR}';done
a Count: 2, Sum: 3, Average: 1.5
b Count: 2, Sum: 7, Average: 3.5
c Count: 1, Sum: 2, Average: 2
d Count: 1, Sum: 5, Average: 5
或者：
# for i in `cat cesc |cut -d, -f1|sort|uniq`;do grep -E ^$i cesc |awk -v i="$i" -F ',' '{sum+=$2} END {print i " Count: " NR ", Sum: " sum ", Average: " sum/NR}';done
a Count: 2, Sum: 3, Average: 1.5
b Count: 2, Sum: 7, Average: 3.5
c Count: 1, Sum: 2, Average: 2
d Count: 1, Sum: 5, Average: 5
 

With python：（python的整形除法預設地板除，只返回一個整形，可以使用from __future__ import division來實現真正的除法）

from __future__ import division
alist = []
blist = []
clist = []
dlist = []
for i in open('cesc'):
    ss = i.split(',')
    if ss[0] == 'a':
        alist.append(int(ss[1]))
    elif ss[0] == 'b':
        blist.append(int(ss[1]))
    elif ss[0] == 'c':
        clist.append(int(ss[1]))
    elif ss[0] == 'd':
        dlist.append(int(ss[1]))
print 'a, Count: ' + str(len(alist)) + ', Sum: ' + str(sum(alist)) + '. Average: ' + str(sum(alist)//len(alist))
print 'b, Count: ' + str(len(blist)) + ', Sum: ' + str(sum(blist)) + '. Average: ' + str(sum(blist)//len(blist))
print 'c, Count: ' + str(len(clist)) + ', Sum: ' + str(sum(clist)) + '. Average: ' + str(sum(clist)//len(clist))
print 'd, Count: ' + str(len(dlist)) + ', Sum: ' + str(sum(dlist)) + '. Average: ' + str(sum(dlist)//len(dlist))

AWK 求和、平均值、最值

記錄幾條命令：（打包目前的目錄下的所有檔案）
ls | awk '{ print "tar zcvf "$0".tar.gz " $0|"/bin/bash" }'

（取範圍）
[root@VM-202 zhuo]# echo "abc#1233+232@jjjj?===" |awk -F '[#@]' '{print $2}'
1233+232

[root@VM-202 zhuo]# echo "abc#1233+232@jjjj?===" |awk -F '[@?]' '{print $2}'
jjjj

awk '/^[^$]/ {print $0}' test.txt        匹配非空行
awk '/^[^zhuo]/ {print $0}' test.txt        匹配非包含zhuo的

替換（將：替換成#）

[root@VM-202 zhuo]# echo "zhuo:x:503:504::/home/zhuo:/bin/bash" |awk 'gsub(/:/,"#") {print $0}'
zhuo#x#503#504##/home/zhuo#/bin/bash

you.txt文檔內容

1
2
3
4

列求和： cat you.txt |awk '{a+=$1}END{print a}'
列求平均值：cat you.txt |awk '{a+=$1}END{print a/NR}'
列求最大值：cat you.txt |awk 'BEGIN{a=0}{if ($1>a) a=$1 fi}END{print a}'

設定一個變數開始為0，遇到比該數大的值，就賦值給該變數，直到結束。
求最小值：cat you.txt |awk 'BEGIN{a=11111}{if ($1<a) a=$1 fi}END{print a}' 反之

_求全文的最值

例：求test.txt的最值
12 34 56 78
24 65 87 90
76 11 67 87
100 89 78 99

for i in `cat test.txt` ;do echo $i; done |sort |sed -n '1p;2p'

例2：同樣是test.txt
求總和：for i in `cat you.txt`;do echo $i ;done |awk '{a+=$1}END{print a}'
________

例3：
A     88
B     78
B     89
C     44
A     98
C     433
要求輸出：A：88；98
          B：78；89
          C：44；433

awk '{a[$1]=a[$1]" "$2}END{for(i in a)print i,a[i]}' test.txt |awk '{print $1":",$2";",$3}'