Multi-stream protocol analysis based on single collector

Streaming (flow) based analysis technology in network industry

There are four kinds of NetFlow, Sflow, Cflow and NetStream. NetFlow is Cisco's unique technology, it is both a traffic analysis protocol, but also a flow-switching technology, as well as the industry's main IP billing method. NetFlow can answer questions about IP traffic, such as who is at what time, where, what protocol to use, who to visit, and how much traffic is. The major versions of the NetFlow protocol are V5, V8 and V9. One of the more widely used is V5 and V8 version. With the advantage of Cisco Network product market share, NetFlow has become the most widely used flow analysis technology nowadays.

Sflow is a kind of network monitoring technology developed by Inmon, HP and foundry networks, it uses data stream random sampling technology, can adapt to traffic analysis under the traffic environment of super large network, so that users can analyze the performance, trend and existing problems of network transmission flow in detail and in real time. Currently, only some of the switches supported by manufacturers such as HP, foundry and Extreme networks support Sflow.

Cflow and NetFlow principle and mechanism are basically the same, it is Juniper Company's unique flow protocol technology.

NetStream is a kind of network flow technology proposed by H3C company, which is similar to the principle of NetFlow technology, but it adds the direction flow statistic function in the realization mechanism (NetFlow only supports flow statistics in the direction). NetStream fully supports multi-core CPU and full distributed processing, which greatly improves the processing ability of machine network flow analysis, and provides a large capacity solution for users to manage, analyze and charge network traffic.

Development trend of flow analysis technology

The traditional NMS based on SNMP can monitor the whole traffic of the network equipment, obtain the real-time or historical inflow/outflow bandwidth, packet loss and error packet of the device port, and can not make further analysis to the specific protocol type and the application flow composition. In the analysis technology based on network probe (PROBE), the interface between "probe" and software is usually private interface, third party software is unusable, and the solution of this kind of hardware and software is usually expensive and deployment is not very convenient. The analysis technology based on real-time grasping package provides detailed protocol analysis data, but lacks the statistic and trend analysis of user traffic access, and can only analyze the packets flowing through the interface in a short time, which cannot meet the requirement of large flow and long time statistics and analysis.

In flow-based streaming analysis, network flows (network flow) are often defined as one-way packets/frame sequences that are transmitted between the source and destination nodes. Usually, network equipment (3 layer switch, router, etc.) itself provides the analysis function based on IP header, is responsible for the analysis and collation of network flow data, according to certain conditions and defined data format to flow collector (stream Collector) output data, Then through the relevant software will collect the flow data collation, analysis and client display. This method based on flow protocol has the advantages of low price and convenient deployment, and can be used for data acquisition and analysis in long time and large flow environment. Recently, IETF technicians are developing the Ipfix (IP flow information Export) specification, which standardizes the format of traffic statistics in the network. Ipfix based on Cisco's NetFlow V9 design, it can set the template format of data output, and it has strong scalability.