Performance Identification of Intrusion Detection Systems

[Introduction]
Performance indicators are a concern of every user when purchasing security products. However, if you do not know the true meaning of these indicators or how these indicators are tested, you will be blinded by the surface parameters to make wrong decisions.

I. Overview

Performance indicators are a concern of every user when purchasing security products. However, if you do not know the true meaning of these indicators or how these indicators are tested, you will be blinded by the surface parameters to make wrong decisions.

This article introduces the meaning and testing methods of the performance indicators of the network intrusion detection system, and analyzes the possible fraud methods during the testing process, in order to provide users with a correct idea of network intrusion detection products.

Ii. Performance indicators

Different security products and performance indicators have different meanings for customers. For example, in the firewall, the customer pays more attention to the throughput per second, the number of concurrent connections per second, and transmission latency. The network intrusion detection system pays more attention to the network data traffic that can be processed per second and the number of network connections that can be monitored per second.

In terms of network intrusion detection systems, in addition to the above indicators, in fact, some indicators that are not known to the customer are also very important, or even more important, for example, the number of packets captured per second and the number of events that can be processed per second.

1. Data traffic per second (Mbps or Gbps)

Data traffic per second refers to the amount of data transmitted through a node per second on the network. This indicator is an important indicator that reflects the performance of the network intrusion detection system, measured in Mbps. For example, 10 Mbps, 100Mbps, and 1 Gbps.

The basic operating principle of the network intrusion detection system is Sniffer. By setting the NIC to the hybrid mode, the NIC can receive all data on the network interface.

If the data traffic per second exceeds the processing capability of the network sensor, NIDS may cause packet loss and thus cannot detect attacks normally. However, whether NIDS will cause packet loss depends not on the data traffic per second, but on the number of packets captured per second.

2. packets captured per second (pps)

Packet Capture per second is the most important indicator of network intrusion detection system performance. Because the system constantly captures packets from the network, analyzes and processes data packets, and finds the intrusion and misuse modes. Therefore, the number of data packets that can be processed per second reflects the system performance. If the industry is not familiar with the intrusion detection system, network traffic per second is often used as a decisive indicator to judge the network intrusion detection system. This idea is wrong. Network traffic per second equals to the number of packets captured per second multiplied by the average size of network packets. Because the average size of network packets varies greatly, the network traffic per second varies greatly with the packet capture rate. For example, if the average size of network data packets is about 1024 bytes and the system performance can support packet capture per second of 10,000 pps, the data traffic that the system can process per second can reach 78 Mbps, when the data traffic exceeds 512 Mbps, packet loss occurs because the system cannot process it. If the average size of the network data packet is about bytes, when the number of packets captured per second reaches 10,000 pps, the data traffic that the system can process per second can reach 40 Mbps. When the data traffic exceeds 40 Mbps, packet loss occurs because the system cannot handle it.

With the same traffic, the smaller the data packet, the more difficult it is to process. Packet processing capability is also the main indicator that reflects firewall performance.

3. Number of network connections that can be monitored per second

The network intrusion detection system not only detects a single packet, but also combines data packets connected to the same network for analysis. The tracking capability of network connections and the ability to reorganize data packets are the basis for protocol analysis and application layer intrusion analysis by the network intrusion detection system. This analysis extends the functions of many network intrusion detection systems, such: detects HTTP attacks, sensitive content detection, email detection, Telnet session recording and playback, and hard disk sharing.

4. Number of events that can be processed per second

After Detecting Network Attacks and suspicious events, the network intrusion detection system generates security events or alarm events and records the events in the event logs. The number of events that can be processed per second reflects the processing capability of the detection and analysis engine and the backend processing capability of Event Logs. Some vendors separate the indicators that reflect the two processing capabilities, known as the performance parameters of the event processing engine and the performance parameters of alarm event records. The performance parameters of alarm events recorded by most network intrusion detection systems are smaller than the performance parameters of the event processing engine, mainly the network intrusion detection system in the Client/Server structure, because of the introduction of network communication performance bottlenecks. In this case, the event is lost, or the console cannot respond.

Iii. What factors are affected by performance indicators?

The performance of the network intrusion detection system depends on the hardware and software.

1. Software Factors

The main software factors are:

● Efficiency of packet capture;
　　
● Efficiency of data packet restructuring and TCP stream restructuring. This is a factor that seriously affects the performance of the network intrusion detection system and imposes a high overhead on the processor and memory. If data packet restructuring and TCP stream restructuring are completed at the user layer of the operating system, the operating system will switch between the core and user States at an extremely high frequency, resulting in a large amount of additional system overhead;

● Efficiency of intrusion analysis. Intrusion detection is generally based on feature matching. network data packets are matched with the features of the intrusion rule repository. Many products use protocol analysis technology to improve the efficiency of intrusion analysis. They first use protocol analysis to filter redundant data, and branch on the Rule tree as soon as possible to accelerate deep traversal;

● Network communication latency in the C/S structure. The network communication module must be introduced to both the server and client to increase the delay of event transmission. Most network intrusion detection systems use the Client/Server structure, such as the ISS Real Secure, Symantec's IDS system, skyrocket and KIDS of jinnuo. For example, some Network Intrusion Detection Systems Based on Browser/Server (B/S) structure do not have this problem. For example, Fang Zheng technology's software uses Sniper because its events are directly stored on network sensors;

● The logging capability of the event logstore. Some systems separate Event Collector from the Event logstore. The Event Collector and Event log database form a C/S structure and introduce latency. If the EC and log database are on different hosts, network transmission latency is introduced. ISS Real Secure, Tianji of Starling and KIDS of jinnuo use this structure, and the network intrusion detection system based on Browser/Server structure does not;

● Event display efficiency on the console. Many consoles cannot handle many events, causing the console to crash. Many C/S-structured consoles provide too many functions, such as network communication with sensors, communication with the Event Collector, and communication with the event log database, event display, event analysis, system management, and configuration are also completed. Introduced many performance bottlenecks. If real-time monitoring is not achieved, the value of the network intrusion detection system will be greatly reduced.

2. Hardware Factors

In terms of hardware, the CPU processing capability, memory, Nic and hard disk I/O are the main features.

● CPU processing capability

CPU processing capability is an important factor affecting the performance of network sensors in the network intrusion detection system. The CPU processing capability has an impact on the system from three aspects: CPU clock speed and number of CPUs, which are called vertical and horizontal CPU scalability. Generally, as the CPU clock speed increases, the processing capability of the network sensor is higher, which is obvious.

But does it linearly increase the performance of network sensors as the number of CPUs increases? It depends on whether the system is multi-process or multi-threaded. Many network intrusion detection systems are optimized with multiple processors.

The utilization of CPU processing capability also greatly affects the performance of network sensors. How can we improve the utilization of CPU processing capability? One of the most important methods is to optimize the CPU Instruction Set of network sensors. For example, on a P4 processor, use the P4 processor's Instruction Set whenever possible. The C & C ++ compiler provided by Intel has the function of optimizing instruction sets, and Intel Labs also provide optimization services in this regard.

Intel's new Xeon processor currently uses hyper-Threading Technology, but network sensors must be optimized for hyper-threading if they want to utilize the performance of the new Xeon processor. At present, the latest Linux core does not support hyper-Threading Technology.

● Memory

The impact of memory on network sensors is significant. Because the network intrusion detection system requires a large amount of memory for packet capture, packet restructuring, stream restructuring, protocol analysis, rule matching, and other calculations.

Memory usage is also crucial because it affects CPU utilization. Usage methods include memory allocation, release, replication, and matching. Improper use may cause memory leakage and occupy CPU overhead.

Some processes of the Network Sensor run in the core state, and the other processes run in the user State. If data is shared between the two processes, memory replication is required, in this case, you need to switch between the core State and the user State. The CPU overhead for switching between the two is very high. If switching is very frequent, the CPU overhead will be very high.

● Secondary Cache

The number of L2 caches also has a positive impact on the performance of network sensors. Therefore, use a large L2 Cache as much as possible.

● Nic

The main impact of NICs on network sensor performance is packet capture efficiency. When the network adapter reaches the peak performance, it is easy to lose packets. Therefore, the NIC of the network sensor cannot use a general Nic. Currently, most network adapters are Intel series and 3Com series. For example, 82559 of Intel's Mbit/s nic and 82543,82544 of gigabit Nic.

If the network intrusion detection system supports multiple network card monitoring, it is best to allocate each network card to different bus segments.

The impact of NICs on the network intrusion detection system is also reflected in the data transmission between network sensors and the console.

Nic drivers also have an important impact on network sensors. Some network intrusion detection systems specifically optimize NICs.

● PCI bus bandwidth

Another important hardware factor is PCI bus bandwidth. Especially in the gigabit network intrusion detection system, to Achieve Packet Capture Rate of several G, multiple 66 Mhz/64-bit PCI or 133 Mhz/64-bit PCI-X bus expansion slots must be used. If you are using a PCI-X bus, you must use a PCI-X-compatible Nic to take full advantage of the PCI-X 133MHz standard. To provide better bandwidth utilization, multiple NICs must be reasonably distributed on the PCI/PCI-X bus segments.

● Hard disk IO

Because the sensors of the network intrusion detection system need to store a lot of log information on the hard disk, the hard disk IO will also affect the performance of the network sensor.
Iv. Analysis

The basic principle of network intrusion detection system performance testing is to use some equipment or software tools to create different packet sizes (such as 64,128,256,512,102 4, 1518 bytes) and background traffic (such as 10 Mbps, 50 Mbps, 100 Mbps, 350 Mbps, 500 Mbps, Mbps, and Mbps) under different pressures, and then launch attacks using various hacker tools, check Network Sensor Detection and packet loss.