Using Logwatch to analyze monitoring log files on Linux

Original: HTTP://SEANLOOK.COM/2014/08/23/LINUX-LOGWATCH-USAGE/1. Introduced

When maintaining a Linux server, it is often necessary to review the logs of various services in the system to check the server's operational status. such as login history, mail, software installation and other logs. It is very inconvenient for the system administrator to check each one, and most of the time, this is a passive check, that is, only when the system is found to be running abnormally, would you want to view the log to get the exception information. Then how to proactively, centrally analyze these logs, and generate reports, scheduled to send to the administrator will appear very important.

Logwatch is an open-source log parsing parser written in the Perl language. It can parse the original log file and convert it into a structured document, as well as customize the report based on your usage and requirements. The main purpose of Logwatch is to generate a more user-friendly log digest, which is not used for real-time processing and monitoring of logs. Because of this, the Logwatch is usually set up with time and frequency automatic timing tasks to be scheduled to run or to be run manually from the command line when log processing is required. Once the log report is generated, Logwatch can send the report to you via email, which you can save as a file or directly on the screen.

The level of detail and reporting coverage of the Logwatch report is fully customizable. Logwatch's log processing engine is also extensible, and in a sense, if you want to use the Logwatch feature in a new application, simply write a log processing script for the application's log file (in Perl) and hook it up to the Logwatch On the line.

Logwatch a little bit bad is that there is no detailed timestamp information in the report it generates, and the original log file exists. You can only know the specific events that have been recorded for a certain period of time, and if you want to know the exact point-in-time information, you have to look at the original log file.

2. Installation and configuration Instructions 2.1 installation

The installation of Logwatch is simple in both the Debian and Redhat systems:

# apt-get install logwatch     //Debian、Ubuntu.etc# yum install logwatch -y       //Redhat、Centos.etc

The following is based on the CentOS 6.x and the rest of the system is small.

2.2 Configuration 2.2.1 Configuration file description

Description of the directory file after installation:

1 2 3 4 5 6 7 8 9 10 11 12 13 14

/usr/share/logwatch default.conf/# Configuration Directory Logwatch.conf# Master profile, recipient, level, etc. logfiles/# define the Log filegroup path for the service to be analyzed, relative to /var/log (*.conf) services/# Customize the Service directory (*.conf) for the log to be analyzed scripts/# executable script logwatch.pl # Start the analysis of the Perl script,/usr/sbin/logwatch the source link logfiles/# Can contain more than one subdirectory of the Logwatch log file group, when the corresponding log service runs, the script under the subdirectory is automatically called services/# Logwatch filtering scripts for the log service, one for shared/# scripts that can be referenced by multiple Logwatch log Services dist.conf/ logfiles/ services/ lib/

By default it is used /usr/share/logwatch/default.conf/logwatch.conf as the master profile, but the /etc/logwatch/conf/logwatch.conf presence configuration option in will overwrite the previous one (or it /usr/share/logwatch logwatch.conf will work, such as an option that is /etc/logwatch logwatch.conf not available in). However, the highest priority is the option specified on the execution command line.

There /etc/logwatch is also a /usr/share/logwatch similar directory structure, where you can add custom monitoring log information.

From the above directory structure can probably understand the principle of Logwatch: Logwatch first to know which service, from the service to get the log file information needed to process, and then this file to filter script processing, after the processing of formatted information to show. For the internal details, see the 3rd article for reference.

2.2.2 Edit Configuration

In the /usr/share/doc/logwatch-7.3.6/HOWTO-Customize-LogWatch file there is detailed configuration instructions here.

Individuals are still accustomed to /etc/logwatch/ managing profiles under, but are not very likely to have two profiles in effect at the same time, so on /usr/share/logwatch/default.conf/logwatch.conf backup, then soft link /etc/logwatch/conf/logwatch.conf :

ln -s /usr/share/logwatch/default.conf/logwatch.conf /etc/logwatch/conf/logwatch.conf

Try to perform logwatch --service sshd --print a feeling of the outcome of the treatment. Next /etc/logwatch/conf/logwatch.conf , modify the file's default configuration to make some personalization settings.

To modify the log analysis level

Detail = <Low, Med, High, or a number>

The "Detail" configuration directive controls the level of detail of the Logwatch report. It can be a positive integer, or it can be a number of high, Med, and low options that represent 10, 5, and 0 numbers, respectively. Set here High . (The configuration file is case-insensitive)

Specify report Recipients

MailTo = youremailaddre[email protected]MailFrom = [email protected]

MailToSpecify Logwatch Log report recipient, to send a report to multiple users, only need to separate their e-mail address with a space or a comma, but logwatch that you have configured the local mail server (SendMail or postfix), and the correct delivery to the user's mailbox.

MailFrom, as the name implies, specifies the sender. The e-mail address can be a complete recipient address, or it can be a local user on the server, such as root (some mail servers do not support displaying sender aliases).

Specify the client that sent the message

mailer = "sendmail -t"

The default is SendMail (not the SendMail server), and is generally fine. In my environment a little special, the mail server must pass SMTP authentication to send mail, not support anonymous and other local MTA post mail, and sendmail I have not found a place to set up SMTP user and password authentication (know the kindly inform), so instead mailer = "mailx -t" , then use the /etc/mail.rcsettings,,,, from smtp smtp-auth-user smtp-auth-password smtp-auth parameters, but the problem with MAILX is that when you set the message report format to HTML, you cannot set the header information so that Foxmail cannot parse the HTML body. Try the SendEmail also not very good solution.

Most people may not be so complex, in fact, is a sender of the function of the client, online learned that there are mutt combined msmtp can solve the problem:

# yum install -y mutt       //mutt其实可以不安装# tar jxvf msmtp-1.4.16.tar.bz2 && cd msmtp-1.4.16# ./configure && make && make install# vi ~/.msmtprc   account default  host your.smtp-server.com  from [email protected]  auth login  user username  password your_auth_pwd  logfile ~/msmtp.log# 如果使用mutt发送，还需要设置~/.muttrc

Change the mailer into a mailer = "msmtp -t" .

Output format

Output = <mail, html or unformatted>

By default, the output format (plain text) is not specified, and the message content that the system administrator sees through the mail client (such as Foxmail) is text-based, simple and bandwidth-saving, and can be specified as a html friendly page with clickable links.

When set at the same time Save = /tmp/logwatch , the Mail report is not sent, and will be Output saved to a file according to the specified format Save .

In addition, in some articles specify the Format option, after I tried in 7.3.6 version invalid.

Collect the range of logs

Range = <Yesterday|Today|All>

RangeConfiguration directives define the time period information for generating Logwatch reports. The general optional value for this directive is yesterday, Today, all. This directive must also be configured when it is in effect, and all Rang = All Archive = yes archived log files (for example,/var/log/maillog,/var/log/maillog-20150111) will be processed.

If we collect it every day through crontab, we can only report yesterday or today's log situation.

Logs of which services are collected

Service = <service-name-1>Service = <service-name-2>. . .

Serviceoption specifies one or more services that you want to monitor. The /usr/share/logwatch/scripts/services services listed under the directory can be monitored, they already cover important system services (e.g. PAM,SECURE,IPTABLES,SYSLOGD, etc.), and also include some like sudo, sshd, HTTP, Fail2ban, Mainstream application services such as Samba. If you want to add a new service to the list, you have to write a corresponding log-processing Perl script and put it in this directory.

For a comprehensive log analysis tool, Logwatch recommends that most people use it Service = "All" , and then by continuing to add Service = "-service_name" and so on to get rid of those non-monitored logs. Of course on the server, not all services under script are started, and some do not have logs.

Command line specify Logwatch options

If you do not want to personalize /etc/logwatch/conf/logwatch.conf , you can leave this file unchanged by default, and then run the following command on the command line as follows:

# logwatch --detail 10 --mailto [email protected] --range today >  

logwatch.confComplete example

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

LogDir =/var/log TmpDir =/var/cache/logwatch Print =No range = yesterday detail = high MailTo = [email protected] mailfrom = [email protected] Span class= "line" >mailer = output = HTML service = All service = service = "-zz-sys" service = "-eximstats"

3. Extended 3.1 cron Daily

We can see the existence of the Crontab timed task set directory /etc/cron.daily/0logwatch :

1 2 3 4 5 6 7 8

#!/bin/bash dailyreport= ' grep -e "^[[:space:]]*dailyreport[[:space:] ]*=[[:space:]]* "/usr/share/logwatch/default.conf/logwatch.conf | head-n1 | Sed -e "s|^\s*dailyreport\s*=\s*| |" ' if [ " $DailyReport "! = " No "] && [ $DailyReport " ! = "no"] then Logwatch fi

If the logwatch.conf option is explicitly set in DailyReport = No , the Logwatch daily execution task is canceled. If you want to modify cron.daily the execution time, you can delete this 0logwatch and add to /etc/crontab , or modify /etc/anacrontab START_HOURS_RANGE .

So Logwatch's work is not to monitor the log after the exception of the timely alarm tool, because the default it is a daily integrated mail, does not have the timeliness (Installation of Perl CPAN module can be more precise control of the logwatch time, see the first reference).

3.2 Customizing the logs you want to monitor

A simple example of how to configure the custom Logwatch is presented.

Create a logwatch log file group first
/etc/logwatch/conf/logfiles/test.conf：

LogFile = /path/to/your/logfileLogFile = /path/to/your/second/logfile

Then create the Logwatch service configuration file
/etc/logwatch/conf/services/test.conf：

Title = test title     # 日志文件里的标题LogFile = test   # logwatch日志文件组的名字，通常是对应的配置文件的文件名部分

Create a Logwatch service filter script
/etc/logwatch/scripts/services/test：

1 2 3

#!/bin/bash Grep-i ERROR

The above script will filter out the rows containing the error from the log file. Finally, add execute permissions for the newly created script:

chmod +x /etc/logwatch/scripts/services/test

Reference

• Using Logwatch monitoring log files in Linux systems (US)
• LogWatch Introduction
• Managing your log files
• Logwatch Simple Configuration Tutorial

Using Logwatch to analyze monitoring log files on Linux