NTOP Monitoring network traffic

Using NTOP to monitor network traffic

____ network traffic reflects the operation status of the network, is the key data to determine whether the network operation is normal, in the actual network, if the network traffic control is not good or network congestion, will lead to network throughput degradation, network performance degradation. The traffic measurement can not only reflect the normal work of network equipment (such as routers, switches, etc.), but also can reflect the resource bottleneck of the whole network operation, so that the management can take the fault remedy and carry out the related business deployment to improve the network performance according to the running state of the network. The network traffic monitoring analysis, can establish a network flow benchmark, through the connection session number tracking, source/Destination address analysis, TCP stream analysis, can be found in the network of abnormal traffic, real-time alarm, so as to ensure network security. The ntop described in this section can provide detailed network traffic schedules. The integrated ntop in the Ossim system can be used directly.

1. NTOP Introduction

____ ntop is a tool for monitoring network traffic, using NTOP to display the network is more intuitive and detailed than some other web-management software. NTOP can even list the network bandwidth utilization of each node computer.

2.Ntop main functions

NTOP mainly provides the following features:

①. Automatically identify useful information from the network;

②. Converting intercepted packets into easy-to-identify formats;

③. Analyze the situation of communication failure in network environment;

④. Detect the communication bottleneck in network environment, and record the time and process of network communication.

____ NTOP can analyze network traffic to determine the various problems on the network, but also can be used to determine whether a hacker is attacking the network system, but also can easily display a specific network protocol, a large number of bandwidth-intensive hosts, the target host of each communication, the sending time of the packet, Details such as the delay of passing the packet.

3. NTOP supported Protocols

____ ntop is easier to install than MRTG, and if you use mobile phone bills to figure out traffic, MRTG is like a phone bill that provides the total cost, while ntop is a breakdown of each fee. At present, the network-managed switches and routers all support SNMP protocol, NTOP supports Simple Network Management protocol, so it can monitor the traffic. NTOP can monitor virtually all protocols on the network: TCP/UDP/ICMP, (R) ARP, IPX, Telnet, DLC, Decnet, DHCP-BOOTP, AppleTalk, Netbios, TCP/UDP, FTP, HTTP, DNS , Telnet, Smtp/pop/imap, SNMP, NNTP, NFS, X11, SSH, and peer-based protocol edonkey.

4.Ntop Support Plugin

①.icmpwatch:

____ For port detection Many people already know that can use Netstat-an to view the current connection and open port, but Netstat is not omnipotent, such as Win2000 by OOB attack, the time of the netstat will have crashed. To this end, there is a special gadget-port listener. Port monitoring is not a complex technique, but it can solve some local problems.

②.netflow:

____ In recent years, many service providers have been using NetFlow. Because the NetFlow is scalable in a large WAN environment, it can help support the best transport streams on peers, and can be used to build infrastructure optimization assessments based on individual services, address service and security issues, and provide the basis for service billing.

③.rrdplugin:

____ is used to generate traffic graphs. RRD, the author, is also the author of Mrtg, RRD can be simply said to be MRTG upgrade version, it is more flexible than MRTG, more suitable for Shell, Perl and other programs to call, genetic the desired picture.

④.sflow:

____ SFlow (RFC 3176) is a standard-based, up-to-date network protocol that addresses many of the problems faced by current network administrators. Sflow has become an "always-on" technology for line-speed operation that embeds Sflow technology in Network routers and switch ASIC chips. Compared to traditional network monitoring solutions using mirrored ports, probes, and bypass monitoring technologies, sflow can significantly reduce implementation costs while enabling a full enterprise network monitoring solution for every port. Unlike packet sampling techniques, such as RMON, Sflow is an export format that adds more information about the monitored packets and forwards the sampled packets using the Sflow agent embedded in the network device, thus exceeding the current RMON, RMON, and functionality in terms of function and performance II and NetFlow technology. Sflow technology is unique in that it can monitor every port in a continuous, real-time manner across the network, but does not require a mirrored monitoring port, and has a very small impact on the overall network performance.

____ In addition, NTOP allows users to install plug-ins to provide reports on specific statistics under specific protocols, such as NFS and NetBIOS plug-ins. Of course, ntop can also generate statistics on the hosts that run it, listing open sockets, incoming and outgoing data, and related host pairs for each process.

Ii. deployment of NTOP systems

For a shared network, the ability to capture network traffic data can be achieved only by placing the network interface connected to the traffic acquisition point in the shared network into a promiscuous mode of operation. Compared with the switching network, when the network congestion, the reliability of the hub network is very low, the SNMP query command and response packets may be delayed or lost, when the ntop detection data is not accurate, for the situation of switched networks, it is necessary to exchange device support (such as a switch with a span port). After the traffic capture host is connected to a port on the switching device, all traffic to be analyzed is mirrored to the acquisition point via the switch's span to (switched Portanalyzer) port. Span is very flexible in use, can monitor a single port on a switch, can monitor multiple ports, and can also monitor VLANs. This makes the flow anomaly monitoring system have a lot of flexibility. In some traffic relatively large enterprises, we generally choose two network card, a network card as ntop dedicated sniffer network card, connect to the core switch mirror Port, another piece with IP address and open the corresponding port (the default is 3000, can also be modified), the role of the connection switch is used to login to the Web interface for management, NTOP is shown in Deployment 1.

Figure 1 Mounting position of the NTOP

____ NTOP does not have its own capture kit, it needs an external packet catcher library: Libpcap. NTOP uses Libpcap to capture packets independently from the physical link, and it can be a true platform-agnostic application with the help of the Libpcap platform. The task of capturing packets directly from the NIC is Libpcap, so we have to make sure that the Libpcap is properly installed under the Linux system.

Three. NTOP installation configuration

____ NTOP work requires the use of zlib, GD, Libpcap and libpng functions, before installation must check whether the server already contains the following software: Zlib (zlib-1.1.3-xx above), GD (gd-1.3.xx above), libpng. You can use RPM to confirm:

Rpm-qa | grep libpcap

Rpm-qa | grep zlib

Rpm-qa | grep GD

Rpm-qa | grep libpng

If you find a missing one, you need to install it yourself, for example.

1. Installing Libpcap

# tar ZXVF libpcap-0.9.8.tar.gz

# CD libpcap-0.9.8

#./configure

# Make&&make Install

2. Installing RRDtool

____ RRDtool refers to the round Robin database tool (ring-shaped databases). Round Robin is a technique for processing quantitative data and pointers to current elements. Imagine a perimeter-labeled ring, which is where the time is stored. Draw an arrow from the center to a point in the circumference, which is the pointer. There is no starting and ending point on a ring and can be stored all the time. After a period of time, all available locations are used, and the loop process automatically reuses the original location. This way, the dataset does not grow and maintenance is not required.

#tar-ZXVF rrdtool-1.3.1.tar.gz

#export pkg_config_path=/usr/lib/pkgconfig/

#./configure

#make

#make Install

3. Installing ntop

Download NTOP installation package: http://www.nmon.net/packages/rpm/x86_64/ntop/

#rpm-IVH ntop-3.3.10-.x86.rpm

#yum Install ntop \\CentOS system

#apt-get Install ntop \\Debian system

____ Note: The NTOP software has been installed for us in the Ossim system and can be used directly. If you choose to install separately, you can continue to refer to the following. Also, if you use red Hat Linux, Fedora, or CentOS, turn off the SELinux feature first.

4. Establish NTOP users and configure permissions

#useradd ntop

5. Create a directory of NTOP storage data

#mkdir-P/var/ntop

#chown-R Ntop.ntop/var/ntop

6. Copying ntop.conf configuration Files

#cp/ntop-3.3.10/ntop.conf.sample/etc/ntop.conf

7. Set Admin password

The administrator password must be established before executing ntop, at least 5 bits in length. Using parameter-A to establish the Administrator password

#ntop-A

8. Ntop Administrator password reset method

ntop user password file is encrypted stored in the Ntop_pw.db file, ntop the user password storage location:

64-bit version:/var/lib/ntop_db_64/ntop_pw.db

The 64-bit version must first delete its password file ntop_pw.db, and then reset the administrator password with NOTP-A, and finally restart the NTOP service to take effect.

#/etc/init.d/ntop restart

Also, note a detail that ntop's access log location is in the/var/log/ntop/directory, and its pcap log is in the/var/lib/ntop directory.

Iv. Application of NTOP

1. Start ntop

#/usr/local/bin/ntop-i eth0-d-l-u ntop-p/var/ntop--use-syslog=daemon

The following are briefly described in the command line.

L-i "eth0": Specifies the listening network card.

l-d: Background execution.

L-l: The output log is written to the system log (/var/log/messages).

L-u ntop: Specifies that execution is performed using ntop identity.

L-p/var/ntop: Specifies the file location of the NTOP database.

L-use-syslog=daemon: Use the System log process.

L-w: Use a different port, specify ntop to use a different port, such as Ntop–w 1900, you can use http://ip:1900 to connect ntop

2. View ntop status with a Web browser

NTOP has a communication port of 3000, so you can see the NTOP Welcome screen when the browser uses ip:3000 to enter NTOP

, shown in 2.

Figure 2 Viewing ntop status

3. View overall traffic

____ for the overall network traffic statistics, respectively is protocol traffic Counters, IP traffic Counters, tcp/udp Connections Stats, Active TCP Connections List, Peers List. According to different Packet, the traffic data is stored in different counter. The overall network traffic is classified and counted, including the following situations.

Traffic distribution situation: Distinguish between this network host, between the network and the external network, the external network and the network traffic statistics between the network.

Packet distribution situation: According to the packet size, broadcast patterns and IP and non-IP, and so on classification and statistics.

Protocol usage and distribution: the type and amount of communication protocol used by each host in the network to transmit and receive data.

With summary→traffic viewing overall traffic (shown in 3), network traffic is displayed in a clear tabular format, as shown in 3.

Figure 3 Viewing overall traffic

____ in Figure 3, the summary content is a general overview of the current play, including traffic, host Network load, and so on. The All protocols option allows you to view the bandwidth consumed by each host and the traffic details used for each period. IP Display Network host traffic status and ranking; Utils can display the network status of NTOP records, traffic statistics and data can be stored in the format txt,xml, plugins contains the type of plug-in supported by NTOP, the admin option can be configured for ntop, For example, we can configure the path of the Pcap log, which is helpful for resolving the disk space problem with ntop data, and the default path is/usr/local/ntop/var/ntop directory. In addition, you can reduce the value of Max hashes and Max sessions in order to conserve disk space. You can also do ntop restart stop. Also, if ntop fails to start, you can look for the error log in/var/log/messages. If you need to set the boot up automatically you can also go to the/etc/rc.d/rc.local file and finally join the command to start ntop. If you want to modify the NTOP appearance you can edit the NTOP HTML document, or the CSS style file, which is in the/usr/share/ntop/html directory.

Figure 4 Displaying network traffic in tabular format

4. View the Communication Packet (protocol) scale

____ packets are of vital importance to network security in the management of networks, such as the role of firewalls to detect packets in a network, to determine if they violate pre-set rules, and to block them if they violate them. The most common packets in a Linux network are TCP and UDP. If you want to know what data is transferred by a computer, you can double-click the computer name to analyze the type of protocol and the percentage of bandwidth consumed by the various network transmissions of the user, as shown in 5.

Figure 5 Viewing the protocol type and occupancy ratio

5. Integration with Google Map: The location of the country where IP is labeled in ntop

____ chose the summary→hosts World Map ntop command to integrate technology with Google Earth, which would display the information collected in real time on Google Earth. First, you need to have a Gmail account and then go to http://code.google.com/apis/maps/signup.html to apply for the Google Maps API key, as shown in the 6 success.

Figure 6 Registering to use the Google Maps API

____ Next copy the key, select Admin→configure→preferences, you will be prompted to enter the user name, password, 7 is shown.

Figure 7 Positioning to admin→configure→preferences

____ Find the Google_maps.key option in the interface shown in 8 and fill in the key . Note that the adjustment parameters need to enter the user and password, if you forget the ntop password, you can enter "/usr/sbin/ntop–a" by root to modify the user admin password.

Figure 8 filling in the key

After saving the exit, select the hosts World Map again in Chrome browser and configure the completion.

Note: Because of Google Maps, you cannot track all IP addresses. If there is a "please enable make sure this ntop html/directory is properly installed" prompt error, most of the permissions issue, can be resolved by the following methods:

#chown-r ntop:ntop/var/lib/ntop/

#chown-r ntop:ntop/usr/share/ntop/

# ln-s/usr/share/ntop/html/var/lib/ntop/

#/etc/init.d/ntop Restart

6. Data dump function

____ Ntop also supports the ability to dump traffic into other formats (such as text files, Perl, PHP, Python) so that other external programs can process the data deeply. You can choose the Utils→data dump command, as shown in 9.

Figure 9 Locating the Utils→data Dump

If we choose to report the host type, the format is PHP. The dump data is as follows:

' 1.1.1.12 ' = Array (

' Hostresolvedname ' = ' 1.1.1.12 ',

' Pktsent ' = 12628,

' Pktrcvd ' = 32668,

' Ipv4bytessent ' = 1818480,

' Ipv4bytesrcvd ' = 30936426,

' Bytesmulticastsent ' = 0,

' Pktmulticastsent ' = 0,

' Bytesmulticastrcvd ' = 0,

' Pktmulticastrcvd ' = 0,

' BytesSent ' = 1818480,

' Bytesrcvd ' = 30936426,

' Ipv4bytessent ' = 1818480,

' Ipv4bytesrcvd ' = 30936426,

' Ipv6bytessent ' = 0,

' Ipv6bytesrcvd ' = 0,

' Tcpbytessent ' = 1813788,

' Tcpbytesrcvd ' = 30936426,

' Udpbytessent ' = 4692,

' Udpbytesrcvd ' = 0,

' Icmpsent ' = 0,

' Icmprcvd ' = 0,

),

7. View network traffic Map (Local networks traffic map)

____ First, in Admin→configure→preference, configure the Dot.path parameter to/usr/bin/dot, and then select Ip→local→network Traffic Map, You can see a response to each host traffic flow topology map, arrow direction represents the flow of data, the mouse click on the corresponding IP address can see very detailed IP statistics. Figure 10 is a topology diagram that ntop automatically generated based on network traffic conditions.

Figure 10Ntop Detection Data Flow chart

8. Viewing host traffic

____ Managers after reviewing the overall network traffic information, but also want to deep analysis of the network host traffic situation, so as to carry out traffic restrictions and other aspects of management, you can choose the ip→summary→traffic,11 shown.

Figure 11 Viewing host traffic

Viewing the session of the transport layer, you can see clearly how many packets were received and sent, as shown in 12.

Figure 12 Viewing the session of the Transport layer

9. With the plugin enabled, NTOP also provides 5 plugins, as shown in 13.

Figure NTOP provides the plugin

(1) Icmpwatch: For port Detection, many people already know can use "Netstat–an" to view the current connection and open port, but Netstat is not omnipotent, in the OOB attack, unequal use of netstat command, the machine has crashed. To this end, there is a special gadget-port listener. Port monitoring is not a complex technique, but it can solve some local problems.

The icon in the ____ diagram indicates that this is a Linux host, and the icon indicates a Windows host that represents a mail server, which represents a Web server. When we need to look at the size of all the servers sent traffic, just click on the sent below byte, if you click on a host under host, you can also display the current host's IP, hostname, MAC, the size of Send/Receive packets per hour, protocol distribution type statistics and other information, 14 , very detailed.

Figure 14

(2) NetFlow: In recent years, many service providers have been using NetFlow. Because the NetFlow is scalable in a large WAN environment, it can help support the best transport streams on peers, and can be used to build infrastructure optimization assessments on a single service basis, address service and security issues, and provide the basis for service billing. NetFlow is a method of data exchange, which works by NetFlow the first IP packet data of the data stream with the standard switching mode, generates NetFlow cache, and then the same data is transmitted in the same data stream based on the cached information, no longer matches the relevant access control policies, The NetFlow cache also contains statistics for subsequent streams.

____ Below we take two steps, first configure a NetFlow forwarding traffic on the router, and then add a NetFlow receive traffic on the NTOP. Enable NetFlow, navigate to Plugins→netflow→activate, then add the device, select the Add NetFlow device option in NetFlow device configuration, 15, set the port to define itself, As long as there is no conflict with the existing, the interface address fills in the network segment address that is intended to be monitored.

Configuration of Figure 15NetFlow

Figure 16 Network Interface selection

____ Then, we need to do the setup on the router, NetFlow was implemented on the router early on, but now some high-end switches support NetFlow, such as the CISCO6500 series.

First, global configuration is required to enable NetFlow:

IP Flow-export version 5

IP flow-sampling-mode packet-interval 100

In the interface that you need to monitor, enable NetFlow:

Interface fastethernet 9/0/1

IP address 192.168.150.20 255.255.255.0

IP Route-cache Flow sampled

Show IP Cache fow//view NetFlow statistics

show IP Flow export//view NetFlow output information

____ Not all NetFlow source devices support interface-based NetFlow, for example Cisco4500 is not supported. That is, it can not be opened in a interface configuration NetFlow, either all ports enabled, or not enabled, it is important to not distinguish the traffic situation on different interface, can only see the entire device traffic situation.

In practice, the following two points need to be noted for configuring NetFlow:

(1) According to the unidirectional nature of the NetFlow flow, the deployment of NetFlow should be based on the network topology as far as possible on the boundary of the two-terminal device configuration protocol.

(2) For Catalyst 60,003-layer switching devices, the Supervisor Engine 1 and Multilayerswitch Feature Card CMSFC Support Multilayer Switching (MLS) for fast switching.

____ Then, is the setup link of the NTOP, which is important, each parameter cannot be set incorrectly. The first is the name of the NetFlow device, you can fill in a random. Next is the port to use, so be sure to fill out the NetFlow application port on the router, for example, 3217. At the same time also for the NetFlow monitoring of the address network segment to do settings, such as the author is 192.168.150.0/255.255.255.0. As shown in 10.24, after each parameter modification, click the right button immediately after completion, navigate to the Admin→switch NIC command in the menu, locate the NetFlow device point switch Nic button that we added to make it effective, we can check the traffic after the effect, As shown in Figure 17.

Figure 17 Viewing traffic

(3) Rrdplugin: Used to generate flow graphs. RRD can be simply said to be MRTG version, it is more flexible than MRTG, more suitable for Shell, Perl and other programs to call, to generate the desired picture.

(4) Sflow:sflow (RFC 3176) is a standard-based, up-to-date network export protocol that solves many of the problems faced by current network administrators. Sflow has become an "always-on" technology for line-speed operation that embeds Sflow technology in Network routers and switch ASIC chips. Compared to traditional network monitoring solutions using mirrored ports, probes, and bypass monitoring technologies, sflow can significantly reduce implementation costs while enabling a full enterprise network monitoring solution for every port. Unlike packet sampling techniques, such as RMON, Sflow is an export format that adds more information about the monitored packets and forwards the sampled packets using the Sflow agent embedded in the network device, thus exceeding the current RMON, RMON, and functionality in terms of function and performance II and NetFlow technology. Sflow technology is unique in that it can monitor every port in a continuous, real-time manner across the network, but does not require a mirrored monitoring port, and has a very small impact on the overall network performance.

(5) Mobile phone plugin: This feature is very interesting, we can use the smartphone, anytime, anywhere monitoring our network, 18 shows.

Figure 18 Phone Plugin

Plugin use HD Demo: http://www.tudou.com/programs/view/Jvq8HOBDOuI/

Application of ntop in virus killing

____ A customer infected with the virus case: one day to work time, network performance suddenly dropped, causing many users can not transfer files on the Internet. The first suspect is the equipment failure, and then found the line whether there is a problem, but can ping the one by one exclusion, and then in the ntop detection of the "IP Protocol" menu to find that the network load is maintained above 95%. In the "Network Traffic:data Sent" chart, one machine in the LAN sends a large number of packets, and this machine's IP address and Mac can also be found. Basically can determine that the virus in the machine sent a large number of UDP packets, resulting in a broadcast storm, resulting in a rapid decline in network performance, 19, is the NTOP captured the randomly sent address list. Find the fault node, and then according to the corresponding mac-ip-wall points, the machine in a timely manner to isolate the network for anti-virus processing.

Figure 19 Virus randomly sending a list of packets

____ In addition, another important feature of ntop is the detection of DDoS type attacks, mainly because it can analyze the network traffic to determine the various problems on the network, but also can be used to determine whether a hacker is attacking the network system, but also can easily display a specific network protocol, occupy a large number of bandwidth of the host, Details such as the target host of each communication, the sending time of the packet, and the delay of transmitting the data packet.

___ above describes some of the features of NTOP tools, but there are many due to the limitations of space is not introduced to everyone, to know that ntop this tool is only a small Ossim platform module, Ossim system integrated NTOP can be NetFlow data into the MySQL database, And can continue to read from the database after the next system boot, without affecting the newly generated data analysis diagram. To find out what Ossim is, please refer to my other relevant blog posts or videos.

NTOP Monitoring network traffic