Log server based on MARIADB and analysis of log management with Loganalyzer

Log files and related service processes

log files are used to record the events that occur during the operation of the system, service, etc., the time of the event, and the critical procedure of the event. The information for these records is used to view the analysis when there is a problem with the server running in order to resolve the issue. In Linux, log records generally have two ways, one is the software itself to complete its own state of record, such as httpd, and the other is provided by the Linux log file management system for unified management. The running software only needs to invoke the related services in this management system to complete the log record. Rsyslog is such a log file management system.

Rsyslog has a lot of facilitybuilt-in that can be understood as services that classify logs from functionality or programs and are responsible for documenting their logs by specialized tools. Software that does not implement the logging feature calls these corresponding service completion logs according to the type of log that needs to be logged.

Rsyslog services are set up mainly in the following:

Auth #记录认证相关的信息

Authpriv #记录认证授权相关的信息

Cron #记录例行性工作cron/at and other information generated

Daemon # records related to each daemon .....

Kern # ... Help Kernel log ...

LPR # ... Print related information .....

Mail # ..... Information about the sending and receiving of mail

Syslog # ..... Information generated by the service itself

News #USENET News Subsystem (below are some of the less familiar ...). ）

User #generic User-level messages

UUCP #UUCP Subsystem

The information generated by each of these services is graded. Some information is only the basic information in the process of system operation, some of which is a major problem in the reporting system. The latter should be taken care of by the system administrator.

Information level:

Debug #debug-level message, debugging information

Info #informational message, basic description information

Notice #normal, but significant, condition, information to follow

Warn, warning #warning conditions, warning message

Err, error #error conditions, incorrect message

Crit #critical Conditions

Alert #action must be taken immediately

Emerg, Panic #system is unusable

configuration file and writing syntax

/etc/rsyslog.conf is the master configuration file for Rsyslog, which records which level of information the facility generates and where it is recorded. The basic syntax of rsyslog.conf is as follows, just give a few examples:

authpriv.*/var/log/secure

This line indicates that all levels of information generated by the AYTHPRIV facility are recorded in/var/log/secure, and "*" is followed by all levels of information, or can be placed in front, for example:

*.emerg *

Represents all the facility generated by the Emerg level and above this level of information, "." Information that represents a level higher than the next (including this level) is recorded. Similar to the following:

1) ".:" #明确指定哪个级别, excluding other levels

2) ".! "#不等于该级别

The "*" on the last side indicates the sending of information to all online people. The location of the log record can also be represented as @192.168.1.110, which means that the log is sent to the remote log server. To record all the information but not include some information, you can write:

*.*;mail.none;authpriv.none;cron.none @192.168.1.110

The above line can also be written as:

*.*;mail,authpriv,cron.none @192.168.1.110

You can also add "-" in front of the location of the log record, which means that asynchronous writes are used to generate more information in some facility, which can improve performance.

Configuration of the log server

Experimental environment

Log server: 192.168.1.110

Database server: 192.168.1.113

httpd Server: 192.168.1.111

PHP Server: 192.168.1.112

On the log server side configuration:

vim /etc/rsyslog.conf# provides udp syslog reception                $ModLoad  imudp                                #启动模块 $UDPServerRun  514                                #监听UDP端口, accept log requests from other servers # provides tcp syslog reception        $ModLoad  imtcp                              $ inputtcpserverrun 514                          #监听TCP端口 

Restart the service to view the ports being monitored:

[[email protected] ~]# service rsyslog restartshutting down system  logger:                                [  OK   ]Starting system logger:                                      [  ok  ][[email protected] ~]# ss  -tuln | grep 514udp    UNCONN     0       0                       *:514                    *:*      udp    UNCONN      0      0                      :::514                   :::*       tcp    LISTEN     0       25                     :::514                   :::*      tcp    LISTEN      0      25                      *:514                    *:*

In the client configuration (192.168.1.104):

Vim/etc/rsyslog.conf*.info;mail.none;authpriv.none;cron.none @192.168.1.110

Information that is generated by all facility (excluding Mail,authpriv,cron) is not recorded locally, but is sent to the log server.

After restarting the service, check the server side for log generation (for example, restart the DNS service, the DNS server is deployed on 192.168.1.104):

[Email protected] ~]# tail/var/log/messages ... Jul 20:41:57 www named[5324]: Zone xiaoxiao.com/in/iplocal:sending notifies (serial 10013) Jul 20:41:57 www named[53 []: Zone Xiaoxiao.com/in/ipother:sending notifies (serial 10006) Jul 20:41:57 www named[5324]: Runningjul 12 20:42:00 CentOS-6 dhclient[1880]: DHCPREQUEST on eth0 to 192.168.1.1 Port (xid=0x3ca6627c)

There are already logs logged to the server.

MARIADB-based log server

Rsyslog also supports logging data to a variety of relational databases, such as MySQL, PostgreSQL, Oracle, and more. Let's take mariadb as an example.

First, install the Rsyslog-mysql software package on the log server, which contains a shared library file (ommysql.so), rsyslog through the shared library file to connect to the MySQL database, and complete the data transfer.

[Email protected] ~]# RPM-QL rsyslog-mysql/lib64/rsyslog/ommysql.so/usr/share/doc/rsyslog-mysql-5.8.10/usr/share/ Doc/rsyslog-mysql-5.8.10/createdb.sql

Execute the SQL statement in Createdb.sql in the MySQL database and complete the database and table that created the Rsyslog dependency.

Initialize the data on the database server (create the corresponding user, authorize):

[[Email protected] ~]# scp 192.168.1.110:/usr/share/doc/rsyslog-mysql-5.8.10/createdb.sql  ./[email protected] ' s password: createdb.sql                                                                                           100% 1046     1.0KB/s    00:00    [[email protected] ~]# mysql < createdb.sql  [[email protected] ~]# mysqlWelcome to the MariaDB monitor.   commands end with&nBSP;;  or \g ...... mariadb [(None)]> show databases;+--------------------+| database            |+--------------------+| information_schema | |  Syslog             |                                    #createDB. SQL-Created database | wordpress           | |  drupal             | |  mysql              | |  performance_schema | |  test               | |  vsftpd_data        |+--------------------+8 rows in set  (0.00 sec) mariadb [(none)] > grant all on syslog.* to [email protected] ' 192.168.%.% '   identified by  ' Rsyslogpass '; query ok, 0 rows affected  (0.00 sec) mariadb [(none)]> flush  privileges; query ok, 0 rows affected  (0.00&NBSP;SEC)

Configure the log server-side rsyslog.conf file to open the ommysql.so module

[Email protected] ~]# vim/etc/rsyslog.conf $ModLoad ommysql......*.info;mail.none;authpriv.none;cron.none:ommysql : 192.168.1.113,syslog,rsysloguser,rsyslogpass

Writing format:

Facility.priority:ommysql:server_ip,database,username,password

Restart the Rsyslog service after configuration is complete. Then restart the DNS service on the client side, generating some logs.

650) this.width=650; "src=" Http://s3.51cto.com/wyfs02/M02/6F/95/wKioL1Wia0LCuS_FAAFj6_ulAoM765.jpg "title=" K7734UKSBBMVWP) ~gifgj{a.png "alt=" Wkiol1wia0lcus_faafj6_ulaom765.jpg "/>

650) this.width=650; "src=" http://s3.51cto.com/wyfs02/M01/6F/98/wKiom1WiaajT46euAAKVKBXg9wY245.jpg "title=" Q} $N {ebav]1fp960tj5lx.png "alt=" Wkiom1wiaajt46euaakvkbxg9wy245.jpg "/>

The log has been recorded in the database ...... ^_^.

Log server based on MARIADB and analysis of log management with Loganalyzer