Detailed Network traffic monitoring

Detailed Network traffic monitoring

The behavior characteristic of the network can be reflected by the dynamic characteristic of the traffic, so it can analyze the operation state of the network from these parameters to monitor the various parameters of the traffic in the network (such as receiving and sending datagram size, packet loss rate, datagram delay, etc.). By analyzing and studying the traffic characteristics carried on the network, it is possible to provide an effective way to explore the internal operation mechanism of the network.

In addition, the network traffic reflects the operation status of the network, is the key to determine whether the network operation is normal. If the network receives more traffic than its actual carrying capacity, it can cause network performance degradation. The flow measurement can not only reflect the normal work of network equipment (such as routers, switches, etc.), but also can reflect the resource bottleneck of the whole network operation. Therefore, the health of network traffic in the enterprise network is just as important as the blood in the human body.

First, the key technology of network monitoring

1. Network Monitoring

Network monitoring is a management tool for monitoring network status, data flow, and information transmission on the network, and its monitoring workflow is: Listeners collect target network segment data stream through a single probe or distributed probe, summarize to remote/Local data center through a predetermined tunnel, and utilize network traffic/ The Protocol analysis system completes the preliminary analysis and preprocessing of the massive data, and finally, according to the task demand, the key data of which is identified, the location and evaluation of the geographical position, to provide the basis for further action. Network monitoring consists of two core technologies, namely data stream acquisition technology and network traffic/Protocol analysis technology. Data stream acquisition refers to the acquisition of data streams from monitored objects (including single or intranet segments) by deploying a network monitoring probe at a specific location, and usually refers to the use of computer AI and intelligence analysis experts to work together to identify the key information needed for a task from a mass of data. and strive to achieve the best balance of efficiency and accuracy.

Network Traffic/Protocol analysis technology can help network operation and maintenance personnel to fully understand and master the network traffic occupancy, application distribution, communication connection, packet original content and all network behavior, as well as the operation of the entire network, so that it can in the network problems, quickly and accurately analyze the cause of the problem, location key points, Fault points and threat points and handle them to ensure that the network is running as intended. It can help us figure out "the details of the operation inside the network."

2. Lack of SNMP protocol

SNMP is the predecessor of the RMON model. Currently, SNMP is based on TCP/IP and is widely used in Internet management protocols, which network administrators can use to monitor and analyze network operation, but SNMP also has some obvious shortcomings. SNMP uses polling to collect data, and polling in large networks generates huge network management messages that can cause network congestion. SNMP provides only general authentication and does not provide reliable security assurances. In addition, SNMP does not support distributed management, but centralized management. Because only the network management station is responsible for collecting data and analyzing data, the processing power of the network management workstation may become a bottleneck. In order to improve the effectiveness of the transmission management message, reduce the load of the network administration workstation, and meet the requirement of monitoring the performance, the IETF has developed rmon to solve the limitation of SNMP in the growing distributed interconnection.

3. Monitoring Key Technologies

The network monitoring system includes two core technologies: Data stream acquisition technology and network traffic/Protocol analysis technology. At the same time, there is another way to divide the industry, the key technology of network monitoring is summarized in the following three aspects:

Data stream acquisition technology solves the problem of how to get the data stream we need from different locations in the network. From the location of data acquisition, can be divided into network-based, host-based and hybrid acquisition of three kinds:

(1) Flow monitoring technology.

The flow monitoring technology mainly includes SNMP-based traffic monitoring and NetFlow-based traffic monitoring. SNMP-based traffic information acquisition. Collect some specific devices and variables related to traffic information by extracting the MIB provided by the network device agent. The network traffic information collected based on SNMP includes the number of bytes lost, the number of broadcast packets, the number of packets lost, and the length of the Output Captain column.

(2) based on NetFlow traffic information collection.

Based on the NetFlow mechanism provided by network equipment, the data collection efficiency and effect can meet the need of network traffic anomaly monitoring. Based on the above flow detection technology, there are many traffic monitoring and management software, this kind of software is an effective tool to judge the flow of abnormal traffic, through the monitoring of traffic size change, can help network management personnel to find abnormal traffic, especially the flow of abnormal traffic flow, so as to further find the source address and destination address of abnormal traffic.

(3) Protocol analysis technology.

Protocol analysis technology is used to solve the understanding of what protocols and applications users use, including protocol and application identification, packet decoding analysis and so on.

4 The difference between NetFlow and sFlow

Current traffic-based solutions are mainly divided into sflow and NetFlow two kinds. Sflow is jointly developed by HP and foundry networks it uses random data stream acquisition technology, can adapt to large network traffic such as in the million-gigabit traffic environment, carry out analysis network transmission, but the support sflow hardware devices are not many, There are currently devices supported by HP and Foundrynetworks and extreme networks manufacturers. NetFlow is Cisco's technology is currently widely used in a variety of medium and high-end devices are supported, but the current support for the million-gigabit traffic is not ideal, it uses timed sampling data acquisition. The NTOP tool's plug-in provides support for Sflow and NetFlow traffic capture.

5. protocol and application identification

According to the content of the data datagram header, using the protocol automata-based traffic recognition technology, the comprehensive analysis includes IP address, port number, key word, message format, transmission layer protocol, etc., classify the traffic and complete the accurate recognition of various application layer protocols, such as database protocol, The use of dynamic port allocation of peer-to, encrypted or non-encrypted instant communication, virtual tunneling applications, etc. will be hidden.

Analysis based on packet decoding. First, the collected data is decoded into a readable data segment according to the message format definition, and then the intelligent state pattern is matched to the massive data segment. The principle of this technique is to decode in the same way as the client or server side of the session, and each protocol component searches for the information pattern after identifying the various parts of the communication data according to the rules defined by the RFC, in some cases by pattern matching in a particular protocol domain, Others need to adopt more advanced techniques or introduce manual intervention, such as testing based on certain variables, such as the length of a domain or the number of independent variables.

6. Network data stream acquisition technology

The best way to control network traffic is to take a full collection of network data streams. At present, there are two kinds of hardware probes and software agents. Network probes (Sensor) typically use devices such as the hub/switch/tap, such as the Common Switch Port Analyzer (SPAN) feature, which is used in the monitoring section of this book, as well as the way in which tap devices are threaded in a network segment; using Hubs (hub) As a network hub switching device, the network is a shared network, and the hub works in a shared bandwidth, and all devices connected to the hub are in a conflict domain, so if the central switching device of the user's network is a hub, you can capture all data traffic in the entire subnet by simply connecting the listening device to the hub.

The Switch Port Analyzer (commonly known as span) is a common and network data stream acquisition port that acts on the switch. The network administrator configures a port on the switch as the span port, and the switch copies and sends the traffic of its specified port/vlan to the span port, which is used to listen for network traffic. Of course, there is a span method also has its shortcomings, it works at the expense of the performance of the switch at the cost of switching (under normal circumstances to enable the switch CPU utilization under 10%, if more than half then can not use the span scheme), in order to solve this problem, In the gigabit rate above the network to try to test the flow of data collection and analysis, it is necessary to use the hardware acceleration technology, at present, the better is the Endace company developed gag series detection card, interested readers can be in-depth query online.

7. limitations of SPAN

• span session;

• 
  

• Generally mid-range Cisco devices typically support only one session;

        requires more than 2 security devices or traffic analysis devices to be used in situations where security levels and requirements are high (for example, multiple IDs systems and multiple traffic analysis systems are used in parallel). Due to the limitation of the number of switch span ports, it is not possible to meet the requirements, so users typically consider using private traffic analysis to access device-TAP (Test access point), while traditional span can be supplemented. Tap-based traffic replication/aggregator, which is a hardware device that supports multi-port traffic aggregation and enables true full-line speeds, which can be fully replicated to multiple listening ports for use by multiple sets of analysis systems. Why it can be so tough, because the TAP device uses a hardware ASIC to replicate the switching engine, so that the gigabit full-speed replication monitoring. It is usually deployed by concatenating the tap device between the firewall and the core switch, and then connecting multiple security devices, such as Ids/ips, to the designated port of tap to achieve the simultaneous operation of several security devices. In the following table 1, let the reader have a clear understanding of the three excellent lack of

Table 1 Comparison of HUB/SPAN/TAP monitoring methods

650) this.width=650; "title=" 4-8-1.jpg "alt=" wkiom1cf9zuiryevaafy9lsaaf4564.jpg "src=" http://s1.51cto.com/wyfs02/ M02/7e/a3/wkiom1cf9zuiryevaafy9lsaaf4564.jpg "/>

         in some large-scale network application enterprises, set up users backstage useIBM WebSphereapplication, when there is a problem, operations personnel will create on multiple switchesSPANPort, we knowCisco6500Series switches can only be set2aSPANPort, if there are multiple sets of monitoring system can not be used at the same time. and cannot be used when the load is largeSPAN, a matrix switch is used to ensure that the monitoring tool is running properly. And more network sniffer tools can be connected to the above analysis. The matrix switch is moreTAPMore is the use of the built-in filtering feature, which allows OPS to select specific data flows over the specified tool. Just imagine the one that can't be filteredTAPinterface, all of a sudden the data from the million-gigabit channel will be washed out. Using the filter function of the matrix switch does not overload the sniffer tool.

Second, analyze network abnormal traffic with NetFlow

As a result of the rapid increase of various network applications, resulting in a proliferation of network traffic. How do Internet users behave in these traffic? How are the various types of traffic distributed? In this case, you can use NetFlow as an effective tool to meet the needs of network traffic management, the tool is NetFlow. Originally NetFlow was developed by Cisco, and because of its wide use, many manufacturers now can implement similar NetFlow functions, such as: Juniper, Extreme, Foundry, h3c. For Cisco, NetFlow has several versions, such as: V5, V7, V8, V9. At present NetFlow V5 is the mainstream. So this article is focused on NetFlowV5, what are the basic elements of this version of the packet, first from flow. For more information, see The open source secure operation dimensional plane-ossim best practices. In the book, not only how to deploy the NetFlow system, how to use it to analyze abnormal traffic, but also the use of another open source tool to analyze the application layer of traffic, and finally in the introduction of how to prevent sniffing technology, fully meet your appetite.

This article is from the "Lee Chenguang Original Technology blog" blog, please be sure to keep this source http://chenguang.blog.51cto.com/350944/1761284

Detailed Network traffic monitoring