Use Ntop to monitor network traffic (video Demo)

Use Ntop to monitor network traffic

____ Network traffic reflects the running status of the network, which is the key data for determining whether the network is running normally. In the actual network, if the network traffic is not well controlled or the network is congested, this will cause a decrease in network throughput and network performance. Traffic measurement not only reflects the normal operation of network devices (such as routers and switches), but also reflects the resource bottlenecks of the entire network, in this way, management personnel can take fault recovery measures and deploy related services in a timely manner based on the running status of the network to improve network performance. Conduct traffic monitoring and analysis on the network, establish a network traffic benchmark, track the number of connection sessions, analyze source/destination addresses, and analyze TCP streams, detects abnormal traffic in the network in a timely manner and generates real-time alerts to Ensure network security. This section describes Ntop to provide a detailed network traffic list. Ntop can be directly used in the Ossim system.

1. Introduction to Ntop

____ Ntop is a tool for monitoring network traffic. Using NTOP to show network usage is more intuitive and detailed than other network management software. NTOP can even list the network bandwidth utilization of each node computer.

2. Main Ntop Functions

Ntop mainly provides the following functions:

①. Automatically identifies useful information from the network;

②. Convert intercepted data packets into a format that is easy to recognize;

③ Analyze communication failures in the network environment;

④. Detect the communication bottleneck in the network environment and record the time and process of network communication.

____ Ntop can identify various problems on the network by analyzing network traffic; it can also be used to determine whether a hacker is attacking the network system; it also allows you to easily display detailed information such as specific network protocols, hosts occupying a large amount of bandwidth, target hosts for various communications, transmission time of data packets, and delay of transmitted data packets.

3. Protocols Supported by Ntop

____ Ntop is easier to install than MRTG. If mobile phone bills are used to compare the traffic, MRTG is like providing the total cost of the phone bill, while Ntop is to list the details of each charge. Currently, network-managed vswitches and vrouters on the market support the SNMP protocol. Ntop supports the Simple Network Management Protocol, so network traffic monitoring can be performed. Ntop can monitor almost all protocols on the Network: TCP/UDP/ICMP, (R) ARP, IPX, Telnet, DLC, Decnet, DHCP-BOOTP, AppleTalk, Netbios, TCP/UDP, FTP, HTTP, DNS, Telnet, SMTP/POP/IMAP, SNMP, NNTP, NFS, X11, SSH, and eDonkey based on P2P technology.

4. Ntop support plug-ins

①. ICMPWATCH:

____ Used for port detection many people already know that they can use NETSTAT-AN to view the current connection and open ports, but NETSTAT is not omnipotent. For example, when Win2000 is attacked by OOB, no more than NETSTAT. To this end, a special tool-the port listener. Port listening is not a complex technique, but it can solve some local problems.

②. NetFlow:

____ In recent years, many service providers have been using NetFlow. Because NetFlow has the scalability in a large WAN environment, it can help to support the best transmission stream on the peer point, and can also be used to optimize the infrastructure evaluation based on a single service, the benefits of solving service and security problems provide the basis for service billing.

③. RrdPlugin:

____ Is used to generate a traffic diagram. The author of RRD is also the author of MRTG. RRD can be simply an upgraded version of MRTG, which is more flexible than MRTG. It is more suitable for calling with shell, perl, and other programs to generate desired images.

④. SFlow:

____ SFlow (RFC 3176) is based on the latest standard network protocol and can solve many problems faced by network administrators. SFlow has become a "always-Online" Technology for line speed operation. It can embed sFlow technology into network routers and switch ASIC chips. Compared with traditional network monitoring solutions that use Image Port, probe, and bypass monitoring technology, sFlow significantly reduces implementation costs, at the same time, it is possible to enable the enterprise-wide network monitoring solution for each port. Unlike the data packet sampling technology (such as RMON), sFlow is an export format that adds more information about the monitored data packets, the sFlow proxy embedded in the network device is used to forward the sampled data packets. Therefore, the functions and performance of the sFlow proxy are superior to the currently used RMON, rmon ii, and NetFlow technologies. The unique feature of sFlow technology is that it can monitor each port in a continuous and real-time manner throughout the network, but does not require Image Monitoring ports, and has little impact on the overall network performance.

____ In addition, Ntop allows users to install plug-ins to provide reports on specific statistics under specific protocols, such as NFS and NetBIOS plug-ins. Of course, Ntop can also generate statistics on the hosts running it, list open sockets, received and sent data, and relevant host pairs for each process.

Ii. Ntop system deployment

For a shared network, you can collect network traffic data only by setting the network interface connected to the traffic collection point in the shared network to a hybrid working mode. Compared with the switch network, when the network is congested, the reliability of the hub network is very low, and the latency or loss of SNMP query commands and response packets may occur. In this case, the Ntop detection data is inaccurate, for switching networks, you need to support switching devices (such as switches with SPAN ports ). After the traffic collection host is connected to a port of the switching device, all traffic images to be analyzed are sent to the collection point through the SPAN to (Switched PortAnalyzer) Port of the switch. SPAN is flexible in use. It can monitor a single port of a vswitch, multiple ports, and VLANs. This makes the traffic exception monitoring system flexible. In some enterprises with relatively high traffic, we generally choose two NICs, one of which serves as the Ntop dedicated sniffer Nic and connects to the mirror port of the core switch, the other part is configured with an IP address and the corresponding port is opened (the default value is 3000, which can be modified). The function of the connection switch is to log on to the Web interface for management, as shown in Ntop deployment 1.

____ Ntop does not have its own packet capture tool. It requires an external packet capture Library: libpcap. Ntop uses libpcap to capture packets independently from the physical link. It can use the libpcap platform to become a truly platform-independent application. Libpcap is responsible for packet capture directly from the NIC. Therefore, we must ensure that libpcap is correctly installed in Linux.

Iii. Ntop installation and configuration

____ Ntop needs to use zlib, gd, libpcap and libpng functions, before installation must check whether the server contains the following software: zlib (zlib-1.1.3-xx or above), gd (gd-1.3.xx or above) and libpng. You can use RPM to confirm:

Rpm-qa | grep libpcap

Rpm-qa | grep zlib

Rpm-qa | grep gd

Rpm-qa | grep libpng

If any one is missing, install it on your own. For example.

1. Install libpcap

# Tar zxvf libpcap-0.9.8.tar.gz

# Cd libpcap-0.9.8

#./Configure

# Make & make install

2. Install RRDtool

____ RRDtool refers to the Round Robin Database tool (ring Database ). Round Robin is a technology used to process quantitative data and current element pointers. Imagine a circle with vertices around it. These vertices are the location of time storage. Draw an arrow from the center of the circle to a point in the circumference. This is the pointer. There is no start point or end point on a ring, and you can store it all the time. After a period of time, all available locations will be used, and the cycle process will automatically reuse the original location. In this way, the dataset does not increase and maintenance is not required.

# Tar-zxvf rrdtool-1.3.1.tar.gz

# Export PKG_CONFIG_PATH =/usr/lib/pkgconfig/

#./Configure

# Make

# Make install

3. Install Ntop

Download ntop installation package: http://www.nmon.net/packages/rpm/x86_64/ntop/

# Rpm-ivh Co., ntop-3.3.10-. x86.rpm

# Yum install ntop \ CentOS System

# Apt-get install ntop \ Debian system

____ Note: Ntop software has been installed for us in the Ossim system and can be directly used. If you choose to install it separately, you can continue to refer to the following content. If you use Red Hat Linux, Fedora, or CentOS, disable SELinux first.

4. Create an Ntop user and Configure permissions

# Useradd ntop

5. Create a directory for Ntop to store data

# Mkdir-p/var/ntop

# Chown-R ntop. ntop/var/ntop

6. Copy the ntop. conf configuration file.

# Cp/ntop-3.3.10/ntop. conf. sample/etc/ntop. conf

7. Set the management password

Before Running ntop, you must create an administrator password with a length of at least five characters. Use parameter-A to create an administrator password

# Ntop-

8. NtopAdministrator Password Reset Method

Ntop user password files are encrypted and stored in the ntop_pw.db file. Ntop user password storage location:

64-bit version:/var/lib/ntop_db_64/ntop_pw.db

In the 64-bit version, you must first Delete the ntop_pw.db password file, then use notp-A to reset the administrator password, and then restart the ntop service to take effect.

#/Etc/init. d/ntop restart

Note that the ntop access log is located in the/var/log/ntop/directory, and its pcap log is located in the/var/lib/ntop directory.

Iv. Application Ntop

1. Start Ntop

#/Usr/local/bin/ntop-I eth0-d-L-u ntop-P/var/ntop -- use-syslog = daemon

The command line is briefly described as follows.

L-I "eth0": Specifies the listening network card.

L-d: executed in the background.

L-L: Write the output log to the System log (/var/log/messages ).

L-u ntop: specifies that Ntop is used for execution.

L-P/var/ntop: Specifies the File Location of the Ntop database.

L-use-syslog = daemon: use the system log process.

L-w: use other ports and specify ntop to use other ports. For example, after running ntop-w 1900, you can use http: // ip: 1900 to connect to ntop.

2. Use a Web browser to view the Ntop status

Ntop communication port is 3000, so you can use IP: 3000 in the browser to enter ntop to see the ntop welcome Interface

, 2.

3. view the overall traffic

____ The overall network Traffic statistics are Protocol Traffic Counters, IP Traffic Counters, TCP/UDP Connections Stats, Active TCP Connections List, and Peers List. Store traffic data to different counters according to different packages. The overall network traffic is classified and statistical, including the following situations.

Traffic Distribution: network traffic statistics between the network host, between the local network and the external network, and between the external network and the local network.

Packet distribution: data packets are classified and analyzed based on the packet size, broadcast format, IP address, and non-IP address.

Protocol usage and distribution: communication protocol types and data transmission volumes used for data transmission and receipt by various hosts in the network.

View the overall Traffic through Summary → Traffic (as shown in 3). The network Traffic is displayed in a clear table, as shown in 3.

____ In figure 3, the Summary content is the overall overview of the game, including traffic and host network load. The All Protocols option allows you to view the bandwidth used by each host and the traffic details used during each time period. The IP address displays the traffic status and ranking of the network host. Utils displays the network status and traffic statistics of the ntop record and stores the data in txt and xml formats; plugins include the plug-in types supported by ntop. The Admin option can be used to configure ntop. For example, we can configure the path of Pcap Log, which is helpful for solving the disk space occupied by Ntop data, the default path is/usr/local/ntop/var/ntop. In addition, the Max Hashes and Max Sessions values can be reduced to save disk space. In addition, you can restart and stop ntop. In addition, if ntop fails to be started, you can find the error log in/var/log/messages. If you need to set Automatic startup, you can add the ntop startup command to the/etc/rc. d/rc. local file. If you want to modify the ntop appearance, you can edit the ntop HTML document or CSS style file in the/usr/share/ntop/html directory.

4. view the communication packet (Protocol) Ratio

____ Data packets are of vital significance for network security management. For example, a firewall is used to detect data packets in the network and determine whether they violate preset rules, in case of violation, block it. The most common data packets in Linux are TCP and UDP. If you want to know what data is transmitted by a computer, double-click the computer name to analyze the protocol types and bandwidth usage of various network transmission protocols, as shown in Figure 5.

5. Integration with Google Map: the location of the country where the IP address is located in Ntop

____ Select the Summary → Hosts World Map Ntop command to integrate the technology with Google Earth to display the collected information on Google Earth in real time. First, you need to have a Gmail account, and then go to http://code.google.com/apis/maps/signup.htmlto apply for the Google Maps API key, as shown in 6 after the success.

____ Next copy the key and select "Admin> Configure> Preferences". A user name and password are displayed, as shown in 7.

____ On the Interface shown in 8, find the google_maps.key option and enter the key. Note: You need to enter the user and password to adjust the parameters. If you forget the Ntop password, you can enter "/usr/sbin/ntop-A" as the root user to change the admin password.

After saving and exiting, select Hosts World Map again in Chrome browser. The configuration is complete.

Note: Due to Google Maps restrictions, you cannot track all IP addresses.

6. Data dumping

____ Ntop also supports traffic dumping into other formats (such as text files, Perl, PHP, and Python), so that other external programs can process data. You can select the Utils> Data Dump command, as shown in figure 9.

For example, select the report host type in PHP format. The dump data is as follows:

'1. 1.1.12 '=> array (

'Hostresolvedname' => '1. 1.1.12 ',

'Pktsent '=> 12628,

'Pktrcvd '=> 32668,

'Listen 4bytessent' => 1818480,

'20174bytesrcvd '=> 30936426,

'Bytesmulticastsent' => 0,

'Pktmulticastsent' => 0,

'Bytesmulticastrcvd '=> 0,

'Pktmulticastrcvd '=> 0,

'Bytedclient' => 1818480,

'Bytesrcvd '=> 30936426,

'Listen 4bytessent' => 1818480,

'20174bytesrcvd '=> 30936426,

'00006bytessent' => 0,

'20176bytesrcvd '=> 0,

'Tcpbytessent '=> 1813788,

'Tcpbytesrcvd '=> 30936426,

'Udpbytessent '=> 4692,

'Udpbytesrcvd '=> 0,

'Icmpsent' => 0,

'Icmprcvd '=> 0,

),

7. view the Network Traffic Map (Local Network Traffic Map)

____ First, Configure dot in Admin → Configure → Preference. the path parameter is/usr/bin/dot, and then select IP → Local → Network Traffic Map to display a topology Map that reflects the Traffic flow of each host, the direction of the arrow indicates the direction of the Data. You can click the corresponding IP address to view the detailed IP statistics. Figure 10 shows the topology automatically generated by Ntop based on network traffic.

8. View host traffic

____ After checking the overall network traffic information, the Administrator also hopes to thoroughly analyze the host traffic in the network and manage traffic restrictions, you can select IP → Summary → Traffic, as shown in 11.

View the session at the transport layer to clearly see the number of packets received and sent, as shown in Figure 12.

9. Enable Plug-ins. Ntop also provides 5 plug-ins, as shown in Figure 13.

(1) ICMPWatch: used for port detection. Many people already know that "netstat-an" can be used to view the current connection and open port, but netstat is not omnipotent, in the case of OOB attacks, the machine will be suspended without using the netstat command. To this end, a special tool-the port listener. Port listening is not a complex technique, but it can solve some local problems.

____ The icon in the figure indicates a Linux host. The icon indicates a Windows host, an email server, and a Web server. If you want to view the sorting of the Sent traffic of all servers, you only need to click the Sent under the Byte. If you click a Host under the Host, the IP address, host name, MAC address, size of packets sent/received per hour, and Protocol distribution statistics of the current host are displayed in detail, as shown in figure 14.

(2) NetFlow: In recent years, many service providers have been using NetFlow. Because NetFlow has the scalability in a large WAN environment, it can help to support the best transmission stream on the peer, and can also be used to optimize the infrastructure evaluation based on a single service, the benefits of solving service and security problems provide the basis for service billing. NetFlow is a data exchange method. its working principle is: NetFlow uses the standard exchange mode to process the first IP packet data of a data stream and generate the NetFlow cache, subsequently, the same data is transmitted in the same data stream based on the cache information, and does not match related access control policies. The NetFlow cache also contains the statistics of subsequent data streams.

____ Let's take two steps: first configure a NetFlow forwarding traffic on the vro, and then add a NetFlow receiving traffic on the Ntop. Enable NetFlow, locate Plugins → NetFlow → Activate, and then Add a Device. In the NetFlow Device Configuration, select the Add NetFlow Device option, as shown in Figure 15. You can set the port as defined by yourself, as long as there is no conflict with the existing one, you can enter the IP address of the network segment to be monitored in the interface address.

____ Next, we need to configure the vro. NetFlow was implemented on the vro in the early days, but some high-end vswitches now support NetFlow, such as Cisco6500 series.

First, you need to configure global settings to enable NetFlow:

Ip flow-export version 5

Ip flow-sampling-mode packet-interval 100

Enable NetFlow on the Interface to be monitored:

Interface FastEthernet 9/0/1

Ip address 192.168.150.20 255.255.255.0

Ip route-cache flow sampled

Show ip cache fow // view NetFlow statistics

Show ip flow export // view NetFlow output information

____ Not all NetFlow source devices support Interface-based NetFlow. For example, Cisco4500 does not. In other words, it cannot enable NetFlow in an Interface configuration, or enable or disable all ports. The important thing is that it cannot distinguish the traffic conditions of different interfaces, only the traffic of the entire device can be seen.

In practice, pay attention to the following two points when configuring NetFlow:

(1) According to the unidirectional nature of the NetFlow, when deploying NetFlow, configure the protocol on the two devices at the boundary according to the network topology.

(2) For Catalyst 6000 layer-3 switching devices, multi-layer switching (MLS) is supported through Supervisor Engine 1 and MultilayerSwitch Feature Card CMSFC.

________Then, It is the Ntop setting step, which is very important. parameters cannot be set incorrectly. First, enter the device name of NetFlow. Next is the port used. Here, you must enter the NetFlow application port on the vro, such as 3217. At the same time, we also need to set the address CIDR Block for NetFlow monitoring. For example, the author uses 192.168.150.0/255.255.255.0. As shown in 10.24, after each parameter is modified and set, click the button on the right to take effect. After the modification is complete, go to the Admin> switch NIC command in the menu, find the added NetFlow device and click the Switch Nic button to make it take effect. After the button takes effect, we can conveniently view the traffic, as shown in figure 17.

(3) rrdPlugin: used to generate a traffic diagram. RRD can be simply an upgraded version of MRTG, which is more flexible than MRTG. It is more suitable for calling with Shell, Perl, and other programs to generate the desired image.

(4) sFlow: sFlow (RFC 3176) is a standard-based latest network export protocol that can solve many problems faced by network administrators. SFlow has become a "always-Online" Technology for line speed operation. It can embed sFlow technology into network routers and switch ASIC chips. Compared with traditional network monitoring solutions that use Image Port, probe, and bypass monitoring technology, sFlow significantly reduces implementation costs, at the same time, it is possible to enable the enterprise-wide network monitoring solution for each port. Unlike the data packet sampling technology (such as RMON), sFlow is an export format that adds more information about the monitored data packets, the sFlow proxy embedded in the network device is used to forward the sampled data packets. Therefore, the functions and performance of the sFlow proxy are superior to the currently used RMON, rmon ii, and NetFlow technologies. The unique feature of sFlow technology is that it can monitor each port in a continuous and real-time manner throughout the network, but it does not require Image Monitoring ports, and has little impact on the overall network performance.

(5) mobile phone plug-in: this function is very interesting. We can use smart phones to monitor our network anytime, anywhere, as shown in 18.

High-Definition demonstration of plug-ins: Http://www.tudou.com/programs/view/Jvq8HOBDOuI/

Application of Ntop in virus detection and removal

____ A customer infected with a virus case: the network performance suddenly dropped during work hours one day, causing many users to be unable to transmit files online. First, I suspected that it was a device fault, and then I found out whether there was a problem with the line, but I was able to ping the device to solve the problem one by one, then, we found that the network load remained above 95% in the "IP protocol" menu of Ntop detection. The "Network Traffic: Data Sent" chart shows that a machine in the lan sends a large number of packets, and the IP address and MAC of the machine can also be found. Basically, we can conclude that the computer has a virus sending a large number of UDP packets, resulting in a broadcast storm, resulting in a rapid decline in network performance, as shown in 19, is the list of random Sending addresses captured by Ntop. Find the faulty node, and then according to the MAC-IP-wall point corresponding, timely isolate the machine out of the network for anti-virus processing.

____ In addition, another important function of Ntop is to detect DDoS attacks. It can analyze network traffic to identify various problems on the network, it can also be used to determine whether a hacker is attacking the network system, it also allows you to easily display specific network protocols, hosts with a large bandwidth usage, target hosts for various communications, the sending time of data packages, and the delay of transmitting data packages.

___ The features of some ntop tools are introduced above. However, many Ntop tools are not introduced due to space limitations. You must know that ntop is only a small module of the Ossim platform, the ntop integrated in the Ossim system can store netflow data into the mysql database and continue to read data from the database after the system starts next time, without affecting the new data analysis graph. For more information about Ossim, see my other blog posts or videos.