Linux network performance monitoring

Network monitoring is the most complex in all Linux subsystems, with too many factors, such as latency, blocking, conflict, and packet loss, even worse, the routers, switches, and wireless signals connected to Linux hosts all affect the overall network and it is difficult to judge whether the problem is caused by the Linux network subsystem or other devices, monitoring and network judgment are the most complex of all Linux subsystems. there are too many factors, such as latency, congestion, conflict, and packet loss, even worse, the routers, switches, and wireless signals connected to Linux hosts all affect the overall network and it is difficult to judge whether the problem is caused by the Linux network subsystem or other devices, it increases the complexity of monitoring and judgment. All the network cards we use now are called adaptive NICS, which means they can be automatically adjusted according to the different network speeds and working modes caused by different network devices on the network. We can use ethtool to view the Nic configuration and working mode:
#/Sbin/ethtool eth0
Settings for eth0:
Supported ports: [TP]
Supported link modes: 10 baseT/Half 10 baseT/Full
100 baseT/Half 100 baseT/Full
1000 baseT/Half 1000 baseT/Full
Supports auto-negotiation: Yes
Advertised link modes: 10 baseT/Half 10 baseT/Full
100 baseT/Half 100 baseT/Full
1000 baseT/Half 1000 baseT/Full
Advertised auto-negotiation: Yes
Speed: 100 Mb/s
Duplex: Full
Port: Twisted Pair
PHYAD: 1
Transceiver: internal
Auto-negotiation: on
Supports Wake-on: g
Wake-on: g
Current message level: 0x000000ff (255)
Link detected: yes

The preceding example shows that the network adapter has three options: 10 baseT, 100 baseT, and 1000 baseT. Currently, the adaptive Speed is 100 baseT (Speed: 100 Mb/s ). You can use the ethtool to force the NIC to work under the 1000 baseT:
#/Sbin/ethtool-s eth0 speed 1000 duplex full autoneg off

Iptraf
The two hosts have network cables (or wireless), routers, switches, and other devices. One way to test the network performance between the two hosts is to send data and collect statistics between the two systems, let's see how throughput, latency, and rate are. Iptraf is a good tool for viewing the network throughput of the local machine. it supports graphic interfaces and is intuitive. The following figure shows that the transmission rate of the Linux system is a little slow under the 100 mbps network, and the Outgoing rates is only 66 mbps.
# Iptraf-d eth0


Netperf
Netperf runs in client/server mode, and can test terminal throughput more diverse than iptraf. Start netserver on the server first:
# Netserver
Starting netserver at port 12865
Starting netserver at hostname 0.0.0.0 port 12865 and family AF_UNSPEC
Then, test the server on the client and run a TCP test lasting 10 seconds:
# Netperf-H 172.16.38.36-l 10
Tcp stream test from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 172.16.38.36 (172.16.38.36) port 0 AF_INET
Recv Send
Socket Message Elapsed
Size Time Throughput
Bytes secs. 10 ^ 6 bits/sec
87380 16384 16384 10.32 93.68
From the above output, we can see that the network throughput is around 94 mbps. for a mbps network, this performance is very good. In the above test, the server and client are located in the same LAN and the LAN is wired. you can also try networks with different structures and different speeds, for example: there are several routers and clients in the middle of the network, such as Wi-Fi and VPN.
Netperf can also establish a TCP connection and send packets sequentially to test the number of TCP requests and responses per second. The following output shows that TCPrequests uses 2 K and responses uses 32 K for processing at a rate of 243 per second:
# Netperf-t TCP_RR-H 172.16.38.36-l 10 ---r 2048,32768
Tcp request/response test from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 172.16.38.36 (172.16.38.36) port 0 AF_INET
Local/Remote
Socket Size Request Resp. Elapsed Trans.
Send Recv Size Time Rate
Bytes Bytes bytes secs. per sec
16384 87380 2048 32768 10.00 243.03
16384 87380

Iperf
Iperf and netperf run in a similar way. They are also in server/client mode. start iperf on the server first:
# Iperf-s-D
------------------------------------------------------------
Server listening on TCP port 5001
TCP window size: 85.3 KByte (default)
------------------------------------------------------------
Running Iperf Server as a daemon
The Iperf daemon process ID: 5695
Then, the client tests the server. the client first connects to the server (172.16.38.36), and tests and samples the network between the server and the client every five seconds within 30 seconds:
# Iperf-c 172.16.38.36-t 30-I 5
------------------------------------------------------------
Client connecting to 172.16.38.36, TCP port 5001
TCP window size: 16.0 KByte (default)
------------------------------------------------------------
[3] local 172.16.39.100 port 49515 connected with 172.16.38.36 port 5001
[ID] Interval Transfer Bandwidth
[3] 0.0-5.0 sec 58.8 MBytes 98.6 Mbits/sec
[ID] Interval Transfer Bandwidth
[3] 5.0-10.0 sec 55.0 MBytes 92.3 Mbits/sec
[ID] Interval Transfer Bandwidth
[3] 10.0-15.0 sec 55.1 MBytes 92.4 Mbits/sec
[ID] Interval Transfer Bandwidth
[3] 15.0-20.0 sec 55.9 MBytes 93.8 Mbits/sec
[ID] Interval Transfer Bandwidth
[3] 20.0-25.0 sec 55.4 MBytes 92.9 Mbits/sec
[ID] Interval Transfer Bandwidth
[3] 25.0-30.0 sec 55.3 MBytes 92.8 Mbits/sec
[ID] Interval Transfer Bandwidth
[3] 0.0-30.0 sec 335 MBytes 93.7 Mbits/sec

TcpdumpAnd tcptrace
Tcpump and tcptrace provide a more detailed analysis method. first, tcpdump captures data packets as required to output the results to a file, and then analyze the file format using tcptrace. This tool combination provides information that is difficult to be found using other tools:
#/Usr/sbin/tcpdump-w network. dmp
Tcpdump: listening on eth0, link-type EN10MB (Ethernet), capture size 96 bytes
511942 packets captured
511942 packets partitioned ed by filter
0 packets dropped by kernel
# Tcptrace network. dmp
1 arg remaining, starting with 'Network. dmp'
Ostermann's tcptrace -- version 6.6.7 -- Thu Nov 4, 2004
511677 packets seen, 511487 TCP packets traced
Elapsed wallclock time: 0:00:00. 510291,100 2714 pkts/sec analyzed
Trace file elapsed time: 0:02:35. 836372
TCP connection info:
1: zaber: 54581-boulder: 111 (a2b) 6> 5 5 3 257 5 255378 5 5 5 5 5 5 5

Tcptrace is very powerful. you can also identify problematic connections by filtering and Boolean expressions. for example, you can find connections with broadcasts greater than 100 segments:
# Tcptrace-f 'rexmit _ segs> 100 'network. dmp

If the connection #10 is faulty, you can view other information about the connection:
# Tcptrace-o10 network. dmp

The following Command uses the slice mode of tcptrace. The program automatically creates a slice. dat file in the current directory, which contains the broadcast information every 15 seconds:
# Tcptrace-monitoring ice network. dmp
# Cat slice. dat
Date segs bytes rexsegs rexbytes new active
---------------------------------------------------------------
16:58:50. 244708 85055 4513418 0 0 6 6
16:59:05. 244708 110921 5882896 0 0 0 2
16:59:20. 244708 126107 6697827 0 0 1 3
16:59:35. 244708 151719 8043597 0 0 0 2
16:59:50. 244708 37296 1980557 0 0 0 3
17:00:05. 244708 67 8828 0 0 2 3
17:00:20. 244708 149 22053 0 0 1 2
17:00:35. 244708 30 4080 0 0 0 1
17:00:50. 244708 39 5688 0 0 0 1
17:01:05. 244708 67 8828 0 0 2 3
17:01:11. 081080 37 4121 0 0 1 3