Introduction to log filtering and Swatch tools

Author: backend
Source: Lance spitzner
Home: http://www.nsfocus.com/
Date:

Filter
Logs have irreplaceable value, but unfortunately they are often ignored, because it is difficult for system administrators to view a large number
Information. Standard log functions cannot automatically filter and Check log records and provide the information required by the system administrator. Below I will briefly describe how to filter and
Obtain the required information and introduce a log tool.
The first part of this article describes how to develop a filtering plan and the required information. The second part is the implementation of log filtering. The log tool described in this article
It is a swatch developed by Todd Atkins.

Start
First, make a plan. Three steps are taken to develop a log plan: the first step is to describe in detail what you need to know and determine what to get from the system log.
The second step is to determine which logs contain the information, and the third step is to develop a trigger that can obtain the information.
For example, if you are very concerned about Sendmail security and want to know if someone is trying to use your email server for spam mail forwarding. In addition
You also want to know if someone tries to obtain related information through SMTP commands (such as EXPN and VRFY) without authorization. These are what we need to determine in the first step.
Content.
The second step is to select a source of information or a log file containing the city information. You can use the system log configuration file/etc/syslog. conf to learn the required information.
Where the information is recorded. For example, the configuration of the email log storage location is as follows:
Homer # Cat/etc/syslog. conf | grep mail
Mail. debug ifdef ('loghost',/var/log/syslog, @ loghost)
The last step is to define the trigger, that is, specify the log information to be recorded. Here we need two triggers for Sendmail.
1. Try to use the email server as an unauthorized IP address of the email forwarder.
2. Try to use the user/or IP address of a disabled command (such as EXPN.
To determine whether a trigger is correctly defined, you can simulate a trigger event and use the/usr/bin/tail-F command to monitor the log file. For the first trigger
The sender (an unauthorized IP address attempts to use the email server for Email Forwarding) can try to use the email server for Email Forwarding from an unauthorized IP address,
You can also run the/usr/bin/tail-F command to view the log records.
Oct 3 14:48:51 Homer Sendmail [6704]: oaa06704:
Ruleset = check_rcpt, arg1 = bsmith@domain.com, relay = foo@moo.com
[206.54.252.1], reject = 550 bsmith@domain.com... relaying denied
We can see that someone is trying to forward emails through our email server on moo.com, which is probably a signal of spam. This is
A log record trigger for unauthorized Email Forwarding. Note that the information also contains IP addresses and domain names.
Now, test the second trigger (using the EXPN command without authorization ). Log on to the SMTP port and run the EXPN command.
/Usr/log/tail-F/var/log/syslog command to view log records.
Oct 2 20:28:37 Homer Sendmail [5453]: noqueue: foo@moo.com [206.54.252.1]: EXPN bsmith [Rejected]
We can see that someone tried to query the user name bsmith on moo.com. This trigger will be activated when you use the EXPN command. Please note that
It also contains IP addresses and domain names.
The above describes three steps to filter logs. First, determine what information is required: unauthorized use of the mail server for mail forwarding and
Use the EXPN command without authorization. Then, determine the logs that contain the information:/var/log/syslog. Finally, verify the log trigger through simulated events
. Now is the time to create an automated filter.

Swatch
Swatch (the simple watcher and filer) is a Perl program developed by Todd Atkins for real-time log monitoring. Swatch benefits
Monitor logging with the specified trigger. When the logging meets the trigger conditions, swatch will notify the system administrator in a predefined manner.
In this example, we need swatch to trigger an alarm when someone attacks Sendmail.
Swatch is very easy to install. It is a Perl program without compilation. Swatch has a very useful installation script,
Copy the manual page and perl file to the corresponding directory. After the installation is complete, you only need to create a configuration file to run the program. Download Swatch
Web: ftp://ftp.stanford.edu/general/security-tools/swatch
The configuration file swatchrc is the focus of the swatch software. This text file tells swatch what logs to monitor and what triggers to search,
And the action to be executed when triggered. When swatch finds that it is consistent with the trigger regular expression defined in swatchrc, it runs in swatchrc
The notification program defined in. Swatch uses/usr/bin/tail-F to monitor log files in real time.
Now let's create a swatchrc file for the sendmail log mentioned above. When someone tries to attack our email server
Sendmail will notify us by email. The content format of the swatchrc file is as follows. It contains four fields separated by tabs, the first two
Fields are required, and the following two fields are optional.
The format of the first field is:
/Pattern/
The "pattern" indicates a regular expression that swatch will search for, that is, our trigger.
The format of the second field is:
Action, action...
The "action" is the action to be executed when the expression matches. Swatch allows you to specify email, call, or any specified execution file.
The format of the Third Field (optional) is:
Hh: mm: SS
HH indicates the number of hours, MM indicates the number of minutes, and SS indicates the number of seconds. This interval is the maximum time for swatch to ignore the same matching expression. For example
If you define a time interval of 5 minutes, swatch will only report the same matching expression once during this time interval, even if the expression may already match
20 times.
The fourth field (required if the third field is used) is a time label in the format of START: length. It defines
The position and length of the time tag in the notification message.
In this Sendmail instance, we will create a swatchrc file containing the matching expression for the above two triggers. We only want
To have an expression match, the system sends a notification email containing the match record to the abuse@ourcompany.com. However, we also need to avoid
Avoid being overwhelmed by excessive warning information. For example, if an attacker attempts to forward 1000 emails within one minute, a large number of notification messages may be generated.
Therefore, we decided to set the interval to 5 minutes. In this way, no matter how many times the same expression is matched within five minutes, only one notification email is sent.
. The following figure shows the final content of the swatchrc file:
/Relaying denied | EXPN/ECHO = normal, mail = abuse@ourcompany.net
The content of the first field is "/relaying denied | EXPN /". If swatch finds that it matches any of these expressions, it will send a warning
Information. The first expression "relaying denied" corresponds to the first trigger we mentioned above. At this time, the log records that someone is trying
Unauthorized Email Forwarding. The second expression "expn" corresponds to the second trigger. At this time, the log records that someone is trying to execute the EXPN command. These two
For details about the trigger, refer to the first part of this article.
The second field contains "Echo = nomal, mail = abuse@company.com ",
Sends an email containing a matching log to the abuse@ourcompany.com.
The content of the third and fourth fields (optional fields) is "", indicating that the same warning message will not be repeated within five minutes, and the time is defined.
The position and length of the tag.
Now we have correctly configured the swatchrc file. The last step is to start the swatch process. Swatch can contain many parameters at startup,
You can start it in the following format:
/Usr/local/bin/swatch-C/var/log/syslogrc-T/var/log/syslog &
-The-C parameter is used to specify the configuration file. The-t parameter specifies the log file to be monitored in real time. "&" enables swatch to run in the background. Swatch generates
Sub-process. Therefore, swatch runs two processes. When stopping swatch, you must kill the two processes. Now, the sendmail log is
Dynamic Filtering. If someone wants to use your Sendmail system at any time, you receive emails containing matching log records.

Summary
Logs are a powerful tool, but a large amount of data can easily overwhelm us. If we do not have enough time to check data in megabytes
May ignore the useful information. The Automatic Log filtering system helps us solve this problem. These automated filtering systems will
The required information notifies us in real time. I hope this article will help you customize your own log file automatic filter.

Source of the original article (Click here)