Rsyslog logs are stored in the mysql database, and loganalyzer is used for web graphical analysis and management.

The importance of system logs, I believe everyone has a deep understanding, when a fault occurs, the first time is to view the relevant error information and log information, to locate the problem, you can also based on the log, analyze logs to obtain the rules of the system running Status. This article describes the first-off content of system logs, which can be divided:

1. Introduction to rsyslog concepts

2. Customize the log storage channel (facility) and storage location so that rsyslog can be used as the server to record the log information of the rsyslog client.

3. Define rsyslog logs to be stored in the mysql database.

4. Use loganalyzer to perform graphical web analysis and management of rsyslog log information stored in mysql databases

Environment:

CentOS7 system, IP address: 10.1.32.72

LAMP combination: yum rpm Package installation

Httpd-2.4

Php-fpm (that is, the implementation of php interaction with httpd based on fastcgi)

Mariadb

Chapter 1 Introduction to rsyslog

1. What is rsyslog?

Rsyslog is a system tool for log management. Its C/S architecture program records system-related log information for the current system, log information can also be recorded for other software or other systems. Its related features are as follows:

Multithreading;

Based on UDP, TCP, and the protocol, you can also encrypt communication based on tls/ssl, or communicate based on RELP;

Stores log information in MySQL, PostgreSQL (PGSQL), Oracle, and other RDBMS;

Powerful filter to filter any part of log information;

Custom output format;

2. rsyslog program environment

Package:

Rsyslog package

Rsyslog-mysql supports logging to mysql packages

Configuration File:

/Etc/rsyslog. conf,/etc/rsyslog. d/*. conf

Main program:

/Usr/sbin/rsyslogd

Module components:

/Usr/lib64/rsyslog/

Unit file:

/Usr/lib/systemd/system/rsyslog. service

3. rsyslog-related terms

Facility: The Channel facility used to collect logs. It can be understood that logs are sent in through that virtual device.

Common facility:

Auth # authentication-related

Authpriv # permission, authorization-related

Cron # Task Scheduler-related

Daemon # daemon

Kern # kernel-related

Lpr # print related

Mail # email-related

Mark # mark related

News # news-related

Security # security-related, similar to auth

Syslog # syslog's own

User # user-related

Uucp # unix to unix cp related

Local0 to local7 # Custom use

* # * Indicates all facility

Priority: (log level) log level, which generally has the following levels (from low to high)

Debug # debugging information of a program or system

Info # General information

Notice # messages that do not affect normal functions

Warning/warn # Important events that may affect system functions

Err/error # error message

Crit # Serious

Alert # must be processed immediately

Emerg/panic # will cause system unavailability

* # Indicates all log levels

None # opposite to *, indicating nothing

Target: (action) location of the log record, common:

File:

Record the specified log information to the specified file.

-File:

Logs are recorded in the specified file.-indicates asynchronous writing.

User:

Notifies a specified user of a log event. Generally, the log information is sent to the terminal of all users logged on to the current system.

Log server:

The format is @ RSYSLOG_SERVER, indicating to send log information to the specified log server.

MPs queue:

The format is | COMMAND, indicating to send the log information pipeline to the specified COMMAND

4. The log format is as follows:

Facility. priority target

Indicates the channel, the log level (the log level above it), and the target to which the logs are uploaded.

Facility format:

* Indicates all facility

FACI1, FACI2, FACI1 ,... All facility given in the list

FACI1.PRI1; FACI2.PRI2; FACI3.PRI3 ;... The given facility and corresponding priority in the list and the above-level records

Priority format:

* All Levels

None has no level and logs are not logged

PRIORITY: this level (inclusive) and above all levels

= PRIORITY: only the specified level

For example:

*. Info; mail. none; authpriv. none; cron. none/var/log/messages

Information above info level of all channels. Logs of all devices except the mail device, authprioriv device, and cron device are recorded in the/var/www/messages file.

Authpriv. */var/log/secure

Logs of all log levels on the authpriv device are recorded in the/var/log/secure file.

Mail. *-/var/log/maillog

Logs of all log levels of the mail device are recorded in the/var/log/mailog file asynchronously.

Auth. = info @ 10.0.0.1

# Indicates to record auth-related logs at the info level to the 10.0.0.1 host.

The premise is that 10.0.0.1 can receive logs sent from other hosts

User .! = Error/var/log/test. log

# Records user-related information, excluding error-level information, recorded in the/var/log/test. log file

Cron.info; mail.info/var/log/test. log

# Multiple log sources can be separated ";"

Cron, mail.info/var/log/test. log

# With cron.info; mail.info

Mail. *; mail .! = Info/var/log/test. log

# Records all mail-related information, but does not include info-level information

Chapter 2 customize the channel (facility) and storage location of log storage so that rsyslog can be used as the server to record the log information of the rsyslog client

 

1. Custom storage location-take the file as an example

     

       

        

    

2. Use the local rsyslog server to accept logs from other rsyslog clients.

In the local rsyslog configuration file, define the attributes required for the server to work.

    

    

        

Configure other rsyslog client hosts to send logs to the rsyslog server.

    

    

        

Verify whether the log information of the client is received on the rsyslog server.

    

Chapter 3 defines that rsyslog logs are stored in the mysql database

1. Install the mariadb database and start the service.

       

    

    

    

    

2. Install rsyslog to connect to the mysql driver module: rsyslog-mysql

    

    

            

2. Import the SQL script generated by rsyslog-mysql to configure the database.

    

    

            

3. Authorize the Syslog Database Management user on the database

    

                

4. Configure rsyslog to use mysql to record logs

    

    

                        

5. After modifying the configuration file, restart the rsyslog service and check whether the database has relevant log information.

    

    

Chapter 4 use loganalyzer to perform web graphical analysis and management of rsyslog log information stored in mysql databases

In the above process, the rsyslog log information has been stored in the mysql database, but it is not convenient to view the log. You need to use SQL statements to query

Next, we will introduce how to use loganalyzer to perform web-based management of logs stored in the mysql database.

1. Deploy the LAMP combination (the combination of php and httpd is fastcgi as an example)

Yum install-y httpd php-fpm php-mysql php-gd

    

            

2. Configure httpd, start the httpd service, configure the php-fpm configuration file, and then start the php-fpm service.

    

    

    

        

    

Test access and verify if LAMP is normal

    

          

          

4. Download, decompress, and deploy the loganalyzer package

    

    

    

5. Log on to the webpage and install

    

    

    

Refresh the webpage after reloading the httpd configuration file

    

    

Wrong shooting process: in the figure above, the database table name defined is different from the actual database (the table name is 'systemevents'). Modify the config. Php file to solve this problem.

Forgot to add session directory to php

Solution: mkdir/var/lib/php/session; chown apache: apache/var/lib/php/session