Network Analysis shell script (real-time traffic + connection statistics)

Network Analysis shell script (real-time traffic + connection statistics)

This article introduces a powerful Analysis Network shell script, which is split from EZHTTP and I think it is necessary to introduce it separately.
Script running effect:

This script provides the following functions:

• 1. monitor traffic of any network card in real time
• 2. Average traffic within 10 seconds
• 3. Calculate the average traffic of each port within 10 seconds based on the statistics of the client and server ports. We can see which ports account for a large amount of traffic. For web servers, port 80 is generally used. When other ports are under attack, the traffic on other ports may be large. Therefore, this function helps us determine whether the port traffic is normal.
• 4. Count the top 10 ip addresses that occupy the maximum bandwidth within 10 s. This feature helps us identify whether there are ip addresses that maliciously occupy bandwidth.
• 5. collect statistics on the connection status. This feature allows us to see which connections are in relatively large states. If the SYN-RECV is in many States, it may be vulnerable to semi-connection attacks. If ESTABLISED is very large, but there are not so many requests through logs, or if tcpdump finds that a large number of ip addresses only establish connections and do not request data, it may be a full connection attack, in this case, if you are using the nginx server, you can add listen 80 deferred in the configuration file to prevent it.
• 6. Check the connection status of each port. This feature helps us find out which port is under attack when it may be attacked.
• 7. The first 10 IP addresses with the maximum number of ESTAB connections in port 80. This feature helps us find out the Ip addresses with too many connections and block them.
• 8. The first 10 IP addresses with the maximum number of SYN-RECV connections in the status of port 80. This function helps us find malicious ip addresses when we are under semi-connected attacks.

Network Analysis tools used:

• 1. tcpdump: This script uses tcpdump to count traffic based on ip addresses or ports.
• 2. ss: This script uses ss commands to calculate the connection status. In actual use, it is much more efficient than netstat.
• 3./proc/net/dev is used to count the traffic of the specified Nic.

Script: https://www.centos.bz/wp-content/uploads/2014/06/network-analysis.sh
The following is a complete script:

1. #! /Bin/bash
3. # Write by zhumaohai (admin # centos. bz)
4. # Author blog: www. centos. bz
7. # Display menu (single choice)
8. Display_menu (){
9. Local soft = $1
10. Local prompt = "which $ {soft} you 'd select :"
11. Eval local arr = (\$ {soft {soft} _ arr [@]})
12. While true
13. Do
14. Echo-e "#####################$ {soft} setting ########### ######### \ n"
15. For (I = 1; I <=$ {# arr [@]}; I ++); do echo-e "$ I) $ {arr [$ I-1]} "; done
16. Echo
17. Read-p "$ {prompt}" $ soft
18. Eval local select =\$ soft
19. If ["$ select" = ""] | ["$ {arr [$ soft-1]}" = ""]; then
20. Prompt = "input errors, please input a number :"
21. Else
22. Eval $ soft =$ {arr [$ soft-1]}
23. Eval echo "your selection: \ $ soft"
24. Break
25. Fi
26. Done
27. }
29. # Converting bandwidth bit units to human readable units
30. Bit_to_human_readable (){
31. # Input bit value
32. Local trafficValue = $1
34. If [[$ {trafficValue %. *}-gt 922]; then
35. # Conv to Kb
36. TrafficValue = 'awk-v value = $ trafficValue 'BEGIN {printf "% 0.1f", value/1024 }''
37. If [[$ {trafficValue %. *}-gt 922]; then
38. # Conv to Mb
39. TrafficValue = 'awk-v value = $ trafficValue 'BEGIN {printf "% 0.1f", value/1024 }''
40. Echo "$ {trafficValue} Mb"
41. Else
42. Echo "$ {trafficValue} Kb"
43. Fi
44. Else
45. Echo "$ {trafficValue} B"
46. Fi
47. }
49. # Determine the package management tool
50. Check_package_manager (){
51. Local manager = $1
52. Local systemPackage =''
53. If cat/etc/issue | grep-q-E-I "ubuntu | debian"; then
54. SystemPackage = 'apt'
55. Elif cat/etc/issue | grep-q-E-I "centos | red hat | redhat"; then
56. SystemPackage = 'yum'
57. Elif cat/proc/version | grep-q-E-I "ubuntu | debian"; then
58. SystemPackage = 'apt'
59. Elif cat/proc/version | grep-q-E-I "centos | red hat | redhat"; then
60. SystemPackage = 'yum'
61. Else
62. Echo "unkonw"
63. Fi
65. If ["$ manager" = "$ systemPackage"]; then
66. Return 0
67. Else
68. Return 1
69. Fi
70. }
73. # Real-time traffic
74. RealTimeTraffic (){
75. Local eth = ""
76. Local nic_arr = ('ifconfig | grep-E-o "^ [a-z0-9] +" | grep-v "lo" | uniq ')
77. Local nicLen =$ {# nic_arr [@]}
78. If [[$ nicLen-eq 0]; then
79. Echo "sorry, I can not detect any network device, please report this issue to author ."
80. Exit 1
81. Elif [[$ nicLen-eq 1]; then
82. Eth = $ nic_arr
83. Else
84. Display_menu nic
85. Eth = $ nic
86. Fi
88. Local clear = true
89. Local eth_in_peak = 0
90. Local eth_out_peak = 0
91. Local eth_in = 0
92. Local eth_out = 0
94. While true; do
95. # Move the cursor to 0: 0
96. Printf "\ 033 [0; 0 H"
97. # Clear the screen and print Now Peak
98. [[$ Clear = true] & printf "\ 033 [2J" & echo "$ eth -------- Now -------- Peak -----------"
99. Traffic_be = ('awk-v eth = $ eth-F' [:] + ''{if ($0 ~ Eth) {print $3, $11} '/proc/net/dev ')
100. Sleep 2
101. Traffic_af = ('awk-v eth = $ eth-F' [:] + ''{if ($0 ~ Eth) {print $3, $11} '/proc/net/dev ')
102. # Computing Rate
103. Eth_in = $ ($ {traffic_af [0]}-$ {traffic_be [0]}) * 8/2 ))
104. Eth_out = $ ($ {traffic_af [1]}-$ {traffic_be [1]}) * 8/2 ))
105. # Calculating traffic peaks
106. [[$ Eth_in-gt $ eth_in_peak] & eth_in_peak = $ eth_in
107. [[$ Eth_out-gt $ eth_out_peak] & eth_out_peak = $ eth_out
108. # Move the cursor
109. Printf "\ 033 [2; 1 H"
110. # Clearing the current row
111. Printf "\ 033 [K"
112. Printf "%-20 s %-20s \ n" "Receive: $ (bit_to_human_readable $ eth_in)" $ (bit_to_human_readable $ eth_in_peak )"
113. # Clearing the current row
114. Printf "\ 033 [K"
115. Printf "%-20 s %-20s \ n" "Transmit: $ (bit_to_human_readable $ eth_out)" $ (bit_to_human_readable $ eth_out_peak )"
116. [[$ Clear = true] & clear = false
117. Done
118. }
120. # Traffic and connection Overview
121. TrafficAndConnectionOverview (){
122. If! Which tcpdump>/dev/null; then
123. Echo "tcpdump not found, going to install it ."
124. If check_package_manager apt; then
125. Apt-get-y install tcpdump
126. Elif check_package_manager yum; then
127. Yum-y install tcpdump
128. Fi
129. Fi
131. Local reg = ""
132. Local eth = ""
133. Local nic_arr = ('ifconfig | grep-E-o "^ [a-z0-9] +" | grep-v "lo" | uniq ')
134. Local nicLen =$ {# nic_arr [@]}
135. If [[$ nicLen-eq 0]; then
136. Echo "sorry, I can not detect any network device, please report this issue to author ."
137. Exit 1
138. Elif [[$ nicLen-eq 1]; then
139. Eth = $ nic_arr
140. Else
141. Display_menu nic
142. Eth = $ nic
143. Fi
145. Echo "please wait for 10 s to generate network data ..."
146. Echo
147. # Current Traffic Value
148. Local traffic_be = ('awk-v eth = $ eth-F' [:] + ''{if ($0 ~ Eth) {print $3, $11} '/proc/net/dev ')
149. # Tcpdump listening Network
150. Tcpdump-v-I $ eth-tnn>/tmp/tcpdump_temp 2> & 1 &
151. Sleep 10
152. Clear
153. Kill 'ps aux | grep tcpdump | grep-v grep | awk '{print $2 }''
155. # Traffic value after 10 s
156. Local traffic_af = ('awk-v eth = $ eth-F' [:] + ''{if ($0 ~ Eth) {print $3, $11} '/proc/net/dev ')
157. # Print the average rate of 10 s
158. Local eth_in =$ ($ {traffic_af [0]}-$ {traffic_be [0]}) * 8/10 ))
159. Local eth_out = $ ($ {traffic_af [1]}-$ {traffic_be [1]}) * 8/10 ))
160. Echo-e "\ 033 [32 mnetwork device $ eth average traffic in 10 s: \ 033 [0 m"
161. Echo "$ eth Receive: $ (bit_to_human_readable $ eth_in)/s"
162. Echo "$ eth Transmit: $ (bit_to_human_readable $ eth_out)/s"
163. Echo
165. Local regTcpdump = $ (ifconfig | grep-A 1 $ eth | awk-F' [:] + ''$0 ~ /Inet addr: /{printf $4 "|"} '| sed-e's/| $ //'-e's/^/(/'-e's/$ /) \\\\\. [0-9] + :/')
167. # The output formats of the new and old versions of tcpdump are different.
168. If awk '/^ IP/{print; exit}'/tmp/tcpdump_temp | grep-q ") $"; then
169. # Process tcpdump files
170. Awk '/^ IP/{print; getline; print}'/tmp/tcpdump_temp>/tmp/tcpdump_temp2
171. Else
172. # Process tcpdump files
173. Awk '/^ IP/{print}'/tmp/tcpdump_temp>/tmp/tcpdump_temp2
174. Sed-I-r's # (. *: [0-9] + \) (. *) # \ 1 \ n \ 2 #'/tmp/tcpdump_temp2
175. Fi
177. Awk '{len = $ NF; sub (/\)/, "", len); getline; print $0, len}'/tmp/tcpdump_temp2>/tmp/tcpdump
179. # Count the average traffic of each port within 10 s
180. Echo-e "\ 033 [32 maverage traffic in 10 s base on server port: \ 033 [0 m"
181. Awk-F' [.:] + '-v regTcpdump = $ regTcpdump' {if ($0 ~ RegTcpdump) {line = "clients>" $8 ". "$9 ". "$10 ". "$11": "$12} else {line = $2 ". "$3 ". "$4 ". "$5": "$6"> clients "}; sum [line] + = $ NF * 8/10} END {for (line in sum) {printf "% s % d \ n", line, sum [line]} '/tmp/tcpdump | \
182. Sort-k 4-nr | head-n 10 | while read a B c d; do
183. Echo "$ a $ B $ c $ (bit_to_human_readable $ d)/s"
184. Done
185. Echo-ne "\ 033 [11A"
186. Echo-ne "\ 033 [50C"
187. Echo-e "\ 033 [32 maverage traffic in 10 s base on client port: \ 033 [0 m"
188. Awk-F' [.:] + '-v regTcpdump = $ regTcpdump' {if ($0 ~ RegTcpdump) {line = $2 ". "$3 ". "$4 ". "$5": "$6"> server "} else {line =" server> "$8 ". "$9 ". "$10 ". "$11": "$12}; sum [line] + = $ NF * 8/10} END {for (line in sum) {printf" % s % d \ n ", line, sum [line]} '/tmp/tcpdump | \
189. Sort-k 4-nr | head-n 10 | while read a B c d; do
190. Echo-ne "\ 033 [50C"
191. Echo "$ a $ B $ c $ (bit_to_human_readable $ d)/s"
192. Done
194. Echo
196. # Count the top 10 ip addresses with the largest bandwidth in 10 s
197. Echo-e "\ 033 [32 mtop 10 ip average traffic in 10 s base on server: \ 033 [0 m"
198. Awk-F' [.:] + '-v regTcpdump = $ regTcpdump' {if ($0 ~ RegTcpdump) {line = $2 ". "$3 ". "$4 ". "$5"> "$8 ". "$9 ". "$10 ". "$11": "$12} else {line = $2 ". "$3 ". "$4 ". "$5": "$6"> "$8 ". "$9 ". "$10 ". "$11}; sum [line] + = $ NF * 8/10} END {for (line in sum) {printf" % s % d \ n ", line, sum [line]} '/tmp/tcpdump | \
199. Sort-k 4-nr | head-n 10 | while read a B c d; do
200. Echo "$ a $ B $ c $ (bit_to_human_readable $ d)/s"
201. Done
202. Echo-ne "\ 033 [11A"
203. Echo-ne "\ 033 [50C"
204. Echo-e "\ 033 [32 mtop 10 ip average traffic in 10 s base on client: \ 033 [0 m"
205. Awk-F' [.:] + '-v regTcpdump = $ regTcpdump' {if ($0 ~ RegTcpdump) {line = $2 ". "$3 ". "$4 ". "$5": "$6"> "$8 ". "$9 ". "$10 ". "$11} else {line = $2 ". "$3 ". "$4 ". "$5"> "$8 ". "$9 ". "$10 ". "$11": "$12}; sum [line] + = $ NF * 8/10} END {for (line in sum) {printf" % s % d \ n ", line, sum [line]} '/tmp/tcpdump | \
206. Sort-k 4-nr | head-n 10 | while read a B c d; do
207. Echo-ne "\ 033 [50C"
208. Echo "$ a $ B $ c $ (bit_to_human_readable $ d)/s"
209. Done
211. Echo
212. # Statistical connection status
213. Local regSS =$ (ifconfig | grep-A 1 $ eth | awk-F' [:] + ''$0 ~ /Inet addr:/{printf $4 "|"} '| sed-e's/| $ //')
214. Ss-an | grep-v-E "LISTEN | UNCONN" | grep-E "$ regSS">/tmp/ss
215. Echo-e "\ 033 [32 mconnection state count: \ 033 [0 m"
216. Awk 'nr> 1 {sum [$ (NF-4)] + = 1} END {for (state in sum) {print state, sum [state]} '/tmp/ss | sort-k 2-nr
217. Echo
218. # Count the connection status of each port
219. Echo-e "\ 033 [32 mconnection state count by port base on server: \ 033 [0 m"
220. Awk 'nr> 1 {sum [$ (NF-4), $ (NF-1)] + = 1} END {for (key in sum) {split (key, subkey, SUBSEP ); print subkey [1], subkey [2], sum [subkey [1], subkey [2]} '/tmp/ss | sort-k 3-nr | head-n 10
221. Echo-ne "\ 033 [11A"
222. Echo-ne "\ 033 [50C"
223. Echo-e "\ 033 [32 mconnection state count by port base on client: \ 033 [0 m"
224. Awk 'nr> 1 {sum [$ (NF-4), $ (NF)] + = 1} END {for (key in sum) {split (key, subkey, SUBSEP ); print subkey [1], subkey [2], sum [subkey [1], subkey [2]} '/tmp/ss | sort-k 3-nr | head-n 10 | awk' {print "\ 033 [50C" $0 }'
225. Echo
226. # Top 10 IP addresses with the maximum number of ESTAB connections in port 80
227. Echo-e "\ 033 [32 mtop 10 ip ESTAB state count at port 80: \ 033 [0 m"
228. Cat/tmp/ss | grep ESTAB | awk-F' [:] + ''{sum [$ (NF-2)] + = 1} END {for (ip in sum) {print ip, sum [ip]} '| sort-k 2-nr | head-n 10
229. Echo
230. # The first 10 IP addresses with the maximum number of SYN-RECV connections in port 80
231. Echo-e "\ 033 [32 mtop 10 ip SYN-RECV state count at port 80: \ 033 [0 m"
232. Cat/tmp/ss | grep-E "$ regSS" | grep SYN-RECV | awk-F '[:] + ''{sum [$ (NF-2)] + = 1} END {for (ip in sum) {print ip, sum [ip]} '| sort-k 2-nr | head-n 10
233. }
235. Main (){
236. While true; do
237. Echo-e "1) real time traffic. \ n2) traffic and connection overview. \ n"
238. Read-p "please input your select (ie 1):" select
239. Case $ select in
240. 1) realTimeTraffic; break ;;
241. 2) trafficAndConnectionOverview; break ;;
242. *) Echo "input error, please input a number .";;
243. Esac
244. Done
245. }
247. Main

If you do not understand the script, leave a message for consultation.

For more information, see https://www.centos.bz/2014/06/shell-script-for-network-analysis/.