Linux server log file search tips

There are dozens of tools used to search for specific activity events in log files. This article describes the policies that should be taken when searching log files. Then, the following examples are provided to illustrate how to manually search log files using the grep command. Next, we will see the usage of the logwatch tool and logsurfer tool. Finally, you will see the tools you need to download and install, such as swatch.

1. Simple Method for searching log files

Generally, system log files are saved in the/var/subdirectory (this path is defined by the syslog. conf file ). If you want all applications to store log files in the/var/subdirectory, edit the configuration files of each application in sequence. It is a good idea to centralize logs into/var/sub-directories. First, you can find all the log files in one place when you need to view them, modify their permissions, or back up them.

Second, the/var/sub-directory is usually in a file system independent of the root directory (/), which helps prevent the log files from rapidly becoming larger and occupying the available space, avoid impact on operating systems and applications. You can use the find command to find the log files you do not know. The specific method is to switch to the root directory first, and then use the root user (root) run the following command to find all recently modified files:

Find.-type f-mtime-5-print | grep-v proc | grep-v lock

2. Policy for searching log files

The first challenge in Log File analysis is to identify abnormal activities from normal activities. To accomplish this challenge, you must know what the normal activities on the system and network look like in the log file. Without accumulated experience, it is difficult to know the performance of regular events in log files. It takes time to familiarize yourself with the normal log file activity. It is unrealistic to understand the log file as soon as it comes up. This is a process that requires time accumulation.

You must know that as the number of applications and users on the network increases or decreases, the content of log files will certainly change accordingly. After the exception is isolated, the next step is to correctly determine whether the exception is an alarm condition. To make a correct judgment, you can only do so by deepening your understanding of the company's daily activities. A balance between system availability and risk prevention is often needed.

3. manually search log files

Grep is one of the most powerful shell commands on Unix systems. Using the grep command to search for various suspicious clues in log files is an excellent use of this text file search command. The usage of the grep command is very simple-input on the command line:

Grep "failed"/var/log/messages

The above grep command will find all text lines containing the word "failed" in the/var/log/messages file. By default, the grep command is case-sensitive. You may need to use the grep command and its "-I" option to perform case-insensitive searches. One of the challenges of searching a log file is that you must first know what you are looking for before you can find out the possibility. There are several ways to help solve this problem.

If you know the event or activity you are looking for-for example, the user tries to use the su command to switch to the root user-you can perform this activity first, then go to the log file to see what it looks like. For example, in SUSE Linux, the failed su command will leave such a record in the log file:

Apr 1 11:15:54 chim su: failed su (to root) rreck on/dev/pts/1


Therefore, if you want to check all such activities, you should use the following command:

Grep "failed su"/var/log/messages

The activities in the above example are a sign of hacker attacks. If the grep Command finds only a few failed events in the log file, it is likely that someone has forgotten the password or encountered an error while typing. Otherwise, if the grep Command finds dozens of such failure events in the log file, it is likely that someone is attempting to break into your system and should immediately take measures to reject their access at the network level.

4. Use logsurfer to search for log files

Logsurfer is a log file search tool. Compared with log search tools such as swatch, logsurfer allows people to make more detailed decisions. Similar to other log search programs, the logsurfer tool matches each line in the log file with some rule expressions. Each matching operation is executed accordingly, those actions are expressed as "rules ". The logsurfer tool is better at some aspects than the swatch tool.

First, the logsurfer tool uses two rule expressions to match text lines. The text lines in the log file must match the first expression, but do not need to match the optional second expression. This kind of arrangement is very useful in some occasions, and it can help people easily eliminate exceptions. Another outstanding advantage of the logsurfer tool is that it can check the context rather than just checking a single text line. This is convenient because a single line of text in a log file is often insufficient to contain the information required to make a decision.

Many people think that the logsurfer tool is difficult to configure because it must be well-versed in Rule expressions and it provides not many configuration examples. If you want to know