Bro: an open source advanced NIDs System

Bro: an open source advanced NIDs System

Creation Time: 2003-10-12
Article attributes: original
Article submitted: stardust (stardust_at_xfocus.org)

Before introducing bro, let's summarize several common open-source NIDS systems:

Snort (http://www.snort.org /)
Currently, the most famous and active open source code NIDs project is located in a lightweight intrusion detection system. It has implemented network detectors and many third-party management and log analysis tools, it is widely used in non-high-speed network environments with low requirements on detection accuracy. Most of the IDS products in China are modified on the basis of their introduction and analysis materials. Snort's success lies in its efficient overall design code, concise and clear rule description design, and a large-scale attack detection rule set, its rule set has been compatible with many other open-source IDs projects.

Prelude IDS (http://www.prelude-ids.org /)
Currently, the active open source code hybrid IDs project focuses more on the modular design of IDS components, and standard idmef format is used for communication between components, from the design perspective, it is positioned to meet the needs of large networks. Currently, network detectors, log analyzer, and alarm information centralized viewing and analysis tools have been implemented. The Network detector is basically a part of the Snort version, fully compatible with the Snort rule set.

Firestorm http://www.scaramanga.co.uk/firestorm/index.html)
The not well-known open source code NIDs project currently only implements the detector part and is fully compatible with the Snort rule set. It also has similar functions and can record the alarm information to the prelude IDs manager, however, it claims that the performance is much better than that of snort.

Netstat (http://www.cs.ucsb.edu /~ RSG/STAT /)
An open-source IDs project in academia that describes attack research results based on Stat (state transition analysis technique, State migration analysis technology) and uses the unique statl language to describe the attack, the attack description text is converted into C ++ code by the statl interpretation tool and compiled into the detection engine to implement the detection function, currently, the statl language translation tool and a basic sample network detector section have been released (a few examples of detection functions ). To use this IDs tool skillfully requires strong programming skills, but using this IDS can implement very complex detection functions.

We can see that most open-source NIDs projects are based on Snort, and some IDs (such as netstat) are mainly used for academic research. Snort is a good way to get started with NIDs technology, IDs in the academic field is mainly used to verify the feasibility of some ideas and concepts. These IDS are generally complicated to use and focus on rigorous and accurate detection. If they are commercialized, great efforts are needed. Snort is easy to implement and easy to use, but it is essentially a sniffer that can match the payload of analysis packets. It is a good network traffic filtering analysis tool for network administrators with rich network knowledge. Because only some simple matching operations can be performed on the payload of TCP/UDP/ICMP packets and some packet attributes, and there is no specific rule association mechanism, Snort can only detect attacks with coarse lines. Currently, the mainstream commercial NIDs products have adopted meticulous application-layer protocol analysis technology. The application-layer protocol analysis technology not only greatly improves the accuracy and efficiency of detection, but also brings the exception detection capability based on protocol analysis, based on rules similar to virus detection, only known attacks can be detected, while exception analysis-based detection may detect some unknown attacks, which is the Development hotspot and direction of New NIDs products, IDS products based on exceptions and misuse will become the mainstream of IDS/IPS products in the future. Although snort performs some simple decoding and processing on some application-layer protocols, such as HTTP, Portmap, and telnet, these processes only optimize the matching, far from protocol analysis, the gap between snort and mainstream IDS products is getting bigger and bigger. Compared with traditional IDS products, Snort is falling behind and learning references are becoming less and less valuable. Is there an even more interesting open-source NIDs project? I have noticed a NIDS software called bro, which provides almost all the desired features.

Next I will introduce bro (http://www.icir.org/vern/bro-info.html), this article aims to analyze bro itself to provide some very interesting and useful features, do not introduce how to install, configure, use, for more information, see the description in the software package.

Bro is a real-time network intrusion detection software implemented by Vern Paxson. It was released in 98 years, it was initially designed to implement a m network with Real-Time alerts, separation of mechanisms and policies, highly scalable intrusion detection, and network monitoring and audit systems.

The system structure of bro is shown as follows:

| Deliver ^ Real-Time
V policy | alert
+ ----------------------------- +
| Policy script interpreter |
+ ----------------------------- +
| Event | rule ^ event
V control | stream
+ -------- + -------------------- +
Rule + -------- V-+ |
-----> | Rule Engine | event generation engine |
+ ---------- + |
+ ----------------------------- +
| Tcpdump ^ filter out
V filter | traffic
+ ----------------------------- +
| Libpcap |
+ ----------------------------- +
^
| Network Packet
+ ----------------------------- +
| Network |
+ ----------------------------- +

The Design and Implementation of BrO follows the layered principle. data packets obtained from the network using libpcap are abstracted into a series of events through the event generation engine and rule engine, these events are further analyzed by the Policy script, and the basic events can trigger alarms. After analyzing the basic events, the policy script can generate new events and trigger alarms. The functions of each component are described in detail as follows. The advanced feature analysis provided by bro is interspersed with the following:

$ Libpcap

Bro uses the libpcap library to obtain network data packets. This allows bro to ignore specific link layer details and achieve maximum portability. Currently, bro supports FreeBSD, Solaris, Linux, Digital UNIX, and other operating systems.

Libpcap obtains the network traffic filter in tcpdump format from the event generation engine at the upper layer, and filters out the traffic that bro is interested in from the NIC for further analysis. For example, the "TCP port 80 or port 23" filter filters TCP data packets whose source port is 80 or 23 from the network for analysis, that is, the analysis target is HTTP and telnet protocol-related data packets.

$ Event generation engine

The data packet obtained from libpcap goes through the event generation engine. The engine first checks the validity of the data packet header to generate a series of events. For example, if the engine receives a TCP packet, it first checks whether the checksum of the TCP header is correct, after the check is passed, the engine checks the flag bit of the header. If the SYN marking engine starts a timer and generates a connection_attempt event, when the engine receives a confirmation packet from the other party, the engine will turn off the timer and generate a connection_established event. If an RST packet is received, the engine will generate a connection_rejected event. All these events are submitted to the policy script interpreter at the upper layer. If you are concerned about these events, you can write a policy script to define corresponding event processing routines for processing.

At this layer, bro further analyzes some application-layer protocols, such as the HTTP protocol, the engine further analyzes the information of data packets, such as requests, responses, access objects, and request methods, and generates corresponding events. For example, the http_request event contains related data fields. Currently, the latest software version supports the analysis of application-layer protocols such as finger, FTP, HTTP, ident, Portmap, and SMTP. More application-layer protocol analysis will be added as software is developed, at this layer, bro implements protocol analysis at the application layer and Provides abstract events to the policy interpreter at the upper layer.

$ Policy script interpreter

Bro policy scripts are used to analyze and process the events generated by the event generation engine. If an attack feature is found to trigger corresponding operations such as alarms, records, or generate new events. Bro's policy script is an analysis script written in bro language that is similar to nfr n-code, the Bro language provides a wide range of data types, process control methods, and many useful analysis functions, which are very similar to or even exclusive to n-code. Such a powerful scripting language can thoroughly analyze abstract events, combined with the application layer protocol analysis and bro script analysis capabilities in the bro event generation engine, it provides very accurate and powerful intrusion detection functions.

The following is a bro script for detecting malicious HTTP requests:

------------------------------------------------------------------------------
# $ ID: http-request.bro, V 1.14 2003/08/31 00:40:21 Vern exp $

# Analysis of HTTP requests.

@ Load HTTP # load the HTTP. Bro script on which this script depends.

Redef capture_filters + = {
["Http-request"] = "tcp dst port 80 or tcp dst port 8080 or tcp dst port 8000"
}; # Define a tcpdump filter to filter HTTP-related traffic

Const sensitive_uris =
/Etc. * //. * (passwd | shadow | netconfig )/
|/Ifs [/T] * =/
|/Nph-test-cgi /? /
|/(% 0a |/./.) // (bin | etc | USR | TMP )/
|/[II] [II] [ss] [AA] [DD] [mm] [PP] [WW] [DD]/
| // (CMD | root | TFTP)/. EXE/
| // Admin_files // order/. log/
| /// Carbo/. dll/
| // Cgi-bin // (PHF | PHP/. cgi | test-cgi )/
| // CGI-dos // ARGs/. BAT/
| // CGI-win // Uploader/. EXE/
| // Search97/. VTS/
& Redef; # use regular expressions to define some common malicious HTTP requests

# Uris that match sensitive_uris but can be generated by worms, and hence
# Shoshould not be flagged (because They're so common ).
Const worm_uris =
/. * // C/+ DIR/
|/. * Cool. dll .*/
|/. * Admin. dll. * admin. dll .*/
& Redef; # define several requests related to SSL-Worm

# Uris that shoshould not be considered sensitive if accessed remotely,
# I. e. by a local client.
Const skip_remote_sensitive_uris: Pattern & redef;
# In order to redef skip_remote_sensitive_uris, You have to also
# Redef the following to T, because there's currently no way in bro
# To specify an empty pattern that can otherwise be used as the default
# For skip_remote_sensitive_uris.
Const have_skip_remote_sensitive_uris = F & redef;

Const sensitive_post_uris =/wwwroot/& redef;

Event http_request (C: connection, method: String, original_uri: String, # handle http_request events
Unescaped_uri: String, version: string) # routine, which can be set for the same event
{# Define multiple processing routines.
Local log_it = f; # The Event generation engine is accompanied by the event
Local uri = unescaped_uri; # For the connection information, please
# Method, request URI, and other words
# Segment information.

If (sensitive_uris in Uri & Uri! = Worm_uris) | # If the URI contains malicious
(Method = "Post" & sensitive_post_uris in URI) # request.
{
If (have_skip_remote_sensitive_uris & # exclude situations that do not need to be concerned.
Is_local_addr (C $ ID $ orig_h )&&
Skip_remote_sensitive_uris in URI)
; # Don't flag it after all
Else
Log_it = T; # Set the record tag.
}

Local S = lookup_http_request_stream (c); # Find the HTTP
# Information. If it does not exist
# Create a network connection phase
# HTTP information. Lookup
# _ Http_request_stream
# The function is in the HTTP. Bro script
# Definition.

If (process_http_replies) # If you want to process HTTP requests
# Response.
{
# To process HTTP replies, we need to record the corresponding
# Requests.
Local n = S $ first_pending_request + S $ num_pending_requests; # Get HTTP related to this connection
# Index number of the information.

S $ requests [N] = [$ method = method, $ uri = Uri, $ log_it = log_it]; # store the relevant HTTP information
# The record indicated by the index number.
++ S $ num_pending_requests;

# If process_http_messages
Local MSG = S $ next_request;

Init_http_message (MSG); # initialize the information of the next record in the pending HTTP Request queue.
# The init_http_message function is defined in the HTTP. Bro script.
MSG $ initiated = T;
}
Else
{
If (log_it) # record.
Alert ([$ Alert = http_sensitiveuri, $ conn = C,
$ MSG = FMT ("% S % s: % S % s ",
Id_string (C $ ID), C $ addl, method, Uri)]);
Print http_log,
FMT ("%. 6f % S % s", network_time (), S $ id, method, Uri );
}
}
------------------------------------------------------------------------------

Writing bro policy scripts requires a lot of time and effort. The author of BrO provides a complete user manual, it includes descriptions of specific data types, descriptions of functions supported by the engine itself, and examples of how to use them. After a certain amount of time, you can fully master them.

$ Rule Engine

The rule engine is a new component feature added to bro 0.8. This feature allows you to define some data packet features as simple rules. The event generation engine matches network data packets based on rules to generate corresponding events, these events may directly generate alert information, or be further analyzed by the Policy script to Form high-level alerts.

Compared with snort rules, bro rules are not only in different formats, but also provide advanced functions not available in Snort rules: matches regular expressions, associations between multiple rules, and interactions between rules and policy scripts. The following example shows the advanced features provided by BrO rules.

The following bro rules can be used to accurately detect two popular attack attempts on rpc dcom overflow vulnerabilities.

Signature rpc-dcom_bind-req {# defines a rule called a rpc-dcom_bind-req that matches an attack session
# The data packet bound to rpc dcom must be prior to the actual attack data packet.
# The matching events do not trigger an alarm independently, which is used for association by the following two rules.
Header IP [] = 6 # The Protocol field matching the IP header is 6, that is, the TCP packet.
Header TCP [2] = 135 # The target port field matching the TCP header is 135, which is also a common attack port of the rpc dcom vulnerability.
TCP-state originator, established # match the TCP packet status to established and sent from the initiator of the connection.
Payload // x05/x00/x0b. {100 ,} /xa0/x01/x00/x00/x00/x00/x00/x00/xc0/x00/x00/x00/x00/x00/x00/x00/x46. */x04/x5d/x88/ x8a/xeb/x1c/xc9/X11/x9f/xe8/x08/x00/x2b/x10/x48/X60/# use regular expressions to match the payload feature of the rpc dcom Bind package
# Sign (RPC header and two specific UUID ).
Event "rpc dcom bind request" # defines the event information that may appear in the alarm log.
}

Signature rpc-dcom_servername-overflow {# define a rule named rpc-dcom_servername-overflow, this rule
# Stack Overflow attacks matching long rpc dcom host names.
Header IP [] = 6
Header TCP [2] = 135
Requires-signature rpc-dcom_bind-req # specifies that the rpc-dcom_bind-req rule must be matched to this rule before
# Yes, that is, this rule is associated with another one in the same TCP session
# Rules: this is a special function provided by BrO rules. Specific to this rule, table
# It means that only after the rpc dcom bind operation is available
# Then match whether there are attack packets in the next packet.
TCP-state originator, established
Payload /. */x05/x00/x00. {100,}/x5c/x00/x5c/x00 [^/] {32,}/# match the payload feature of the attack data packet, with positive
# Expression check whether there are hosts with more than 32 bytes
# Name.
Event "rpc dcom servername Stack Overflow Attempt"
}

Signature rpc-dcom_pathname-overflow {# define a rule named rpc-dcom_pathname-overflow, this rule
# A heap overflow attack that matches the long path name of rpc dcom.
Header IP [] = 6
Header TCP [2] = 135
Requires-signature rpc-dcom_bind-req # the same premise as the preceding rule.
TCP-state originator, established
Payload /. */x05/x00/x00. {100,}/x5c/x00/x5c/x00 [^ //] {5, 32}/x5c/x00. {520 ,} /x00/x00/# match attack data
# Package payload feature
# Sign, expressed using regular expressions
# Check whether there is an exceeded number
# Long Path Name.
Event "rpc dcom pathname Heap Overflow Attempt"
}

The above bro rules demonstrate the powerful features of rules that support regular expression matching and rule Association. Regular Expressions can precisely match the characteristics of attack data. Of course, regular expressions are also inefficient, but for engines that focus on detection accuracy for hundreds of megabytes of network traffic, this price is acceptable and acceptable. Rule Association is actually a very powerful feature: the association between rules can greatly improve the accuracy of detection, such as the above example of detecting rpc dcom attacks; you can associate an attack request and response data to determine whether the attack is successful. For example, you can associate a CGI Attack request with the HTTP status code in its response packet to determine whether the attack is successful; by associating certain event sequences, You can trigger more advanced events. For example, you can associate multiple malicious CGI requests to draw a conclusion that the system is being attacked by a CGI vulnerability scan.

The following bro rules used to detect SSL-worm attacks demonstrate the interaction between rules and policy scripts.

Signature sslworm-probe {# define a rule named sslworm-probe to detect SSL-worm scan operations
Header IP [] = 6 # TCP packet
Header TCP [2] = 80 # TCP destination port 80
Payload/. * Get // http // 1/. 1/x0d/x0a/x0d/x0a/# match the characteristics of SSL-worm scan data in payload
Event "host may have been probed by Apache/SSL worm" # define alarm information in log records
}

Signature sslworm-vulnerable-probe {# defines the rules for scanning vulnerable hosts by associating the scan detection rules above
Requires-signature sslworm-probe # reference the scan detection rules above
Eval sslworm_is_server_vulnerable # executes the pre-defined function in the bro policy script. This function is used to determine
# Disconnect the OpenSSL software version of the host to determine whether the attacker has scanned
# A vulnerable host is found.
Event "host may have been probed by Apache/SSL worm and is vulnerable" # alarm information
}

The above rules show the interaction between rules and policy scripts. From rules, you can reference the functions in the Policy script to complete specific functions, the function returns the success or failure information after the function is executed to the rule. The rule determines whether to trigger related events based on the returned information. The function referenced in the rule can be used to determine certain matching conditions and return results. The above SSL-worm example also provides more in-depth analysis and Association of multiple events; you can even use the function functions referenced in the rule to perform response operations, such as disconnecting.

Conclusion: compared with the simple and rough snort detection method, bro provides some key advanced features: the application layer protocol function implemented in the event generation engine, powerful rule scripts used for in-depth analysis of events, data matching and rule association that support regular expressions (Rules of the same session are directly associated, cross-session rule association can be implemented using policy scripts), interaction between rules and policy scripts, these features make the open-source NIDs System of BrO more similar to the current mainstream commercial NIDs products. After fine tuning, it can fully implement accurate detection of many commercial NIDs products, bro is a good tool for NIDs developers to refer to and learn from. It is also a good tool for users with experience in network management to customize some attack detection functions.