Build remote system log collection and analysis environment based on CentOS 6.5+mariadb+loganalyzer+syslog

first, to understand the meaning of the Rsyslog configuration file

Configuration file Path/etc/rsyslog.conf

In Rsyslog facility facilities can be used to classify logs from functions or programs in the following ways

Auth	and certification-related
Authpriv	Related to the certification authority
Cron	Specifically for the periodic task schedule to be logged
Daemon	Log records for the daemon
Kern	Logging to the kernel
Lpr	Logging for printing
Mail	Logging messages
Mark	Firewall tags
News	Newsgroups (this is very early)
Security	Security-related (equivalent to auth)
Syslog	Own log
User	and user-related
Uucp	Log of copy files between UNIX hosts
Local	Custom facility from Local0 through to LOCAL7 8 reserved

The above facilities can be used with wildcards in the specified facility, and the available wildcard characters are as follows

*	means all facility.
,	Can be the meaning of multiple facility lists, such as Facility1,facility2,facility3
!	The meaning of the inverse is expressed in addition to its

Below are the log levels, from top to bottom, from low to high, and with less information logged

Debug	With the information of the tune, the log information is the most, the most detailed information
Info	Log of general information, most commonly used
Notice	Information of the most important general conditions
Warn,warning	Notice that needs attention, and at this point it needs attention.
Err,error	Critical error, information that prevents a feature or module from working correctly
Alert	Information that is equivalent to a blue alert that prevents the entire system or the entire software from working properly
Crit	The equivalent of Orange alert, need to immediately modify the repaired warning message
Emerg,panic	It's the red alert, and when this happens, the kernel hangs up a second.
None	Don't do anything, don't record anything.

These levels can also be used with wildcards, and the available wildcard characters have the following

*	All the Levels
None	Without any level

Here's how to configure a remote MARIADB (MySQL)-based storage log system

1, we need to modify the/etc/rsyslog.conf configuration file.

First we enable # # # MODULES # # # #Port listening in the segment configuration file

yum install rsyslog-mysql -y
sed -ri ‘s / ^ # (\ $ ModLoad im (tc | ud) p) / \ 1 /‘ /etc/rsyslog.conf
sed -ri ‘s / ^ # (\ $ (InputTCP | UDP) ServerRun) / \ 1 /‘ /etc/rsyslog.conf
Then add a MariaDB (MySQL) -enabled configuration to the MOUDLES section of /etc/rsyslog.conf
sed -i ‘/ InputTCPServerRun / a \\ n \ n # Log event to MariaDB \ n \ $ ModLoad ommysql \ n’ /etc/rsyslog.conf

650) this.width=650; "class=" Alignnone size-full wp-image-633 "src=" http://www.05hd.com/wp-content/uploads/2014/08/ Rsyslog2014-08-30-164010.png "alt=" rsyslog2014-08-30 164010 "width="/"height=" "389" border= "0" hspace= "0" vspace= "0 "Title=" "style=" width:750px;height:389px;/>

2, and then I go to collect logs on the server compiled install MARIADB do log collection storage

[[email protected] ~] # yum -y groupinstall "Development tools" "Server Platform Development"
[[email protected] ~] # yum -y install libxml2-devel cmake
### Above is the necessary software for compiling
[[email protected] ~]# wget http://mirrors.hustunique.com/mariadb/mariadb-10.0.12/source/mariadb-10.0.12.tar.gz
[[email protected] ~]# tar xf mariadb-10.0.12.tar.gz
[[email protected] ~]# cd mariadb-10.0.12
[[email protected] mariadb-10.0.12]# make clean
[[email protected] mariadb-10.0.12]# cmake . -DCMAKE_INSTALL_PREFIX=/usr/local/mysql -DMYSQL_DATADIR=/mydata/data -DSYSCONFDIR=/etc -DWITH_SSL=system -DWITH_SPHINX_STORAGE_ENGINE=1 -DWITH_INNOBASE_STORAGE_ENGINE=1 -DWITH_ARCHIVE_STORAGE_ENGINE=1 -DWITH_BLACKHOLE_STORAGE_ENGINE=1 -DWITH_READLINE=1 -DWITH_ZLIB=system -DWITH_LIBWRAP=0 -DMYSQL_UNIX_ADDR=/tmp/mysql.sock -DDEFAULT_CHARSET=utf8 -DDEFAULT_COLLATION=utf8_general_ci -DWITH_ARIA_STORAGE_ENGINE=1 -DWITH_XTRADB_STORAGE_ENGINE=1 -DWITH_INNOBASE_STORAGE_ENGINE=1 -DWITH_PARTITION_STORAGE_ENGINE=1 -DWITH_FEDERATEDX_STORAGE_ENGINE=1 -DWITH_MYISAM_STORAGE_ENGINE=1 -DENABLED_LOCAL_INFILE=1 -DWITH_EMBEDDED_SERVER=1
[[email protected] mariadb-10.0.12]# make -j 4 && make install
[[email protected] mariadb-10.0.12]# cd ~/
###Here are the necessary configuration files and environment variables to modify
[[email protected] ~]# cp /usr/local/mysql/support-files/my-large.cnf /etc/my.cnf
[[email protected] ~]# cp /usr/local/mysql/support-files/mysql.server /etc/rc.d/init.d/mysqld
[[email protected] mariadb]# chmod +x /etc/rc.d/init.d/mysql
[[email protected] mariadb]# chkconfig --add mysql
[[email protected] mariadb]# chkconfig --list mysql
mysql 0:off 1:off 2:on 3:on 4:on 5:on 6:off
[[email protected] ~]# sed -i "/^thread_concurrency.*/a datadir=/mydata/data" /etc/my.cnf
[[email protected] ~]# groupadd mysql
[[email protected] ~]# useradd -g mysql -r -s /sbin/nologin -m /mydata/data mysql
[[email protected] ~]# chown -R mysql.mysql /mydata
[[email protected] ~]# sed -i "$(cat /etc/man.config | grep -nE ‘^MANPATH[[:space:]]+‘ | tail -1 | awk -F: ‘{print$1}‘)a MANPATHt/usr/local/mysql/man" /etc/man.config
[[email protected] ~]# echo "export PATH=/usr/local/mysql/bin:$PATH" > /etc/profile.d/mariadb1002.sh
[[email protected] ~]# /usr/local/mysql/scripts/mysql_install_db --user=mysql --datadir=/mydata/data/
[[email protected] mariadb]# cp /usr/local/mariadb/support-files/mysql.server /etc/rc.d/init.d/mysql
[[email protected] mariadb]# service mysql start
Starting MySQL..... [ OK ]
[[email protected] mariadb]# service mysql stop
Shutting down MySQL. [ OK ]
[[email protected] ~]# service mysqld start

Do database import again

yum install rsyslog-mysql -y
[[email protected] ~]# find / -name rsyslog-mysql*
/usr/share/doc/rsyslog-mysql-5.8.10

650) this.width=650; "class=" Alignnone size-full wp-image-625 "src=" http://www.05hd.com/wp-content/uploads/2014/08/ Rsyslog2014-08-30-171545.png "alt=" rsyslog2014-08-30 171545 "width=" 543 "height="/>

Let's import the Createdb.sql from this directory into the database

Mysql-uroot-p </usr/share/doc/rsyslog-mysql-5.8.10/createdb.sql

The following is the SQL command

MariaDB [(none)]> grant all on Syslog.* to [email protected]‘localhost‘ identified by ‘rsyslog用户的密码‘;
MariaDB [(none)]> flush privileges;

Finally, we go back to the server that needs to log remote collection statistics
Modify the configuration file again
650) this.width=650; "class=" Alignnone size-full wp-image-630 "src=" http://www.05hd.com/wp-content/uploads/2014/08/ Rsyslog2014-08-30-172615.png "alt=" rsyslog2014-08-30 172615 "width="/"height=" "429" border= "0" hspace= "0" vspace= "0 "Title=" "style=" width:750px;height:429px;/>

Now we can configure the Loganalyzer on the MARIADB (MySQL) server

Loganalyzer need to match Apache or Nginx Web server (this article does not introduce more)

wget http://www.05hd.com/wp-content/uploads/2014/08/loganalyzer-3.6.5.tar.gz
### Official Download Page http://loganalyzer.adiscon.com/downloads/
[[email protected] ~] # wget http://www.05hd.com/wp-content/uploads/2014/08/loganalyzer-3.6.5.tar.gz
--2014-08-31 02: 39: 01-- http://www.05hd.com/wp-content/uploads/2014/08/loganalyzer-3.6.5.tar.gz
Resolving www.05hd.com ... 2001: 41d0: 8: ea32: 1234: 5678: 9abc: 9999, 5.135.167.50
Connecting to www.05hd.com | 2001: 41d0: 8: ea32: 1234: 5678: 9abc: 9999 |: 80 ... connected.
HTTP request sent, awaiting response ... 200 OK
Length: 1046957 (1022K) [application / octet-stream]
Saving to: “loganalyzer-3.6.5.tar.gz”

100% [=================================================== ===================================================>] 1,046,957 --.- K / s in 0.09s

2014-08-31 02:39:02 (11.2 MB / s)-"loganalyzer-3.6.5.tar.gz" saved [1046957/1046957]

[[email protected] ~] # tar xf loganalyzer-3.6.5.tar.gz
[[email protected] ookBack ~] # cp -a loganalyzer-3.6.5 / src / * /home/wwwroot/rsyslog.05hd.com/
[[email protected] ~] # cp -a loganalyzer-3.6.5 / contrib / * /home/wwwroot/rsyslog.05hd.com/
[[email protected] ~] # chown -R /home/wwwroot/rsyslog.05hd.com/
[[email protected] ~] # chmod + x /home/wwwroot/rsyslog.05hd.com/*.sh
[[email protected] ~] # /home/wwwroot/rsyslog.05hd.com/configure.sh
[[email protected] ~] # service rsyslog restart 

This is the time to open a Web page to configure Loganalyzer on WebUI.

Build remote system log collection and analysis environment based on CentOS 6.5+mariadb+loganalyzer+syslog