Rsyslog+mysql+loganalyzer Deploying a log server

Experimental requirements

1. Build a visual log collection and analysis platform for centralized collection of logs, and through the visualization of log analysis tools to present;

2. Apps server is to collect the log nodes, can have more than one, here I only use 1 node;

3. Rsyslog server is unified to receive each nodes submitted to the log, monitoring in the TCP/UDP 514 nodes;

4. MySQL server is used to store the submitted log information, as a standalone server, can also do cluster;

5. Loganalyzer is a lamp-based visual log analysis tool, back-end query database, the results collated output;
   

The topology is as follows:

650) this.width=650; "src=" Http://s3.51cto.com/wyfs02/M00/75/62/wKioL1Y4QdqQYF0qAAjhMN3X8AA023.bmp "title=" Drawing13.bmp "alt=" Wkiol1y4qdqqyf0qaajhmn3x8aa023.bmp "/>

Implementation mechanism

RSYSLOGD on Linux can submit generated logs to a remote log server, Rsyslog server can not only write logs to local files, but also write logs to the database through the Rsyslog-mysql module. Loganalyzer can analyze the log information and visualize the appearance, obviously by querying the MySQL database stored in the log information can be completed!

Lab Step 1. Deploying MySQL Server

1.1 install MySQL

  Here will not repeat the installation process, refer to   mariadb Universal binary Deployment manual

1.2 Creating a Rsyslog dependent database

Because Rsyslog and MySQL are separated in the schema, MySQL must have a specific database if it wants to accept a specific format of log information from Rsyslog server, and the tables in the database have specific fields to receive specific log information. Of course it doesn't need us to define it. The Rsyslog-mysql module has its own CREATEDB.SQL statement that automatically creates a specific database.

Yum-y Install rsyslog-mysqlmysql-h localhost-u ROOT-PZXCZXC </usr/share/doc/rsyslog-mysql-5.8.10/createdb.sql

The exact location of this SQL execution script may be different, and the location can be queried using Rpm-ql/rsyslog-mysql!

1.3 Authorizing the Syslog database

By default MySQL does not allow connections from remote, here we create a dedicated user to have full administrative rights to the Syslog database, and can be connected from remote

Mysql>grant all privileges in syslog.* to ' abc ' @ ' 10.134.140.65 ' identified by ' zxczxc '; mysql>flush privileges;

650) this.width=650; "src=" http://s3.51cto.com/wyfs02/M01/75/62/wKioL1Y4RlHyYcgjAAG9VCEVLxk027.jpg "title=" Image 6.png "alt=" Wkiol1y4rlhyycgjaag9vcevlxk027.jpg "/>

Lab Step 2. Deploying Rsyslog Server

1.1 Installing the RSYSLOGD, installing the Rsyslog-mysql module

The default major release has the main program installed, no need to install, if not #yum-y install RSYSLOGD

Yum-y Install Rsyslog-mysql

1.2 Editing the master configuration file

vim/etc/rsyslog.conf#### MODULES # # # # # $ModLoad Onmysql---> Load connection MySQL database module authpriv.*: Onmysql:10.134.140.65,syslog , Abc,zxczxc$modload Imudp$udpserverrun 514---> Monitor collects nodes logs in udp514

650) this.width=650; "src=" http://s3.51cto.com/wyfs02/M01/75/65/wKiom1Y4SMGjadrSAAKAlOnykGQ635.jpg "title=" Image 8.png "alt=" Wkiom1y4smgjadrsaakalonykgq635.jpg "/>

1.3 Restart Service can be

Service Rsyslog Restart

Experiment Step 3. Deploy nodes

The so-called nodes, that is, the need to collect information servers, only need to edit the master configuration file to

Vim/etc/rsyslog.conf in the last line, add the log classifications that you want to collect and submit to the log server. Level authpriv.* @10.134.140.64

Save and restart the service!

note that the meaning of this is to submit the authpriv.* log information to the log server 10.134.140.64, the destination port is the udp514 configured in step 2 by default.

1. rsyslog.conf The default configuration options log collection will still take effect! Just save in the local, if you do not want to save the local, only need to comment out the line, only keep @10.134.140.64;

2. The collection of log information is specified in facility.priority, such as authpriv.* indicates that all levels of the authentication authorization class are collected

The manual can be used for specific classifications and levels.

Experiment Step 3. Simple test

Now a simple set of log collection model is completed, all the nodes on the Login authentication authorization log information will be submitted to Rsyslog Server, and write to the database, then we try to log 10.134.140.63 the machine, and then look at the database!

3.1 Analog 2-time error login

650) this.width=650; "src=" Http://s3.51cto.com/wyfs02/M02/75/65/wKiom1Y4TFOSX8c2AAU88DgnSpI893.bmp "title=" Baidushurufa_2015-11-3_13-54-59.bmp "alt=" Wkiom1y4tfosx8c2aau88dgnspi893.bmp "/>

3.2 Querying the MySQL database

For the sake of visualization, I used the navicat for MySQL connection tool to view

650) this.width=650; "src=" http://s3.51cto.com/wyfs02/M02/75/63/wKioL1Y4TTqSknZVAAR05D9yFyg923.jpg "title=" Image 13.png "alt=" Wkiol1y4ttqsknzvaar05d9yfyg923.jpg "/>

Success, then we just need to integrate Loganalyzer!

Lab Step 4. Integrate Loganalyzer and test

Loganalyzer itself is written in PHP language, need PHP engine, obviously to and Apache, and log source on the back-end MySQL server, so need Php-mysql driver, so directly build a set of lamp environment can. You can refer to the post to compile and install lamp

Here I use Yum directly to install!

4.1 Installing the Lamp environment

Yum-y Install httpd php php-mysql PHP-GD

I here yum source for Aliyun Centos-6, installed lamp version is as follows

650) this.width=650; "src=" http://s3.51cto.com/wyfs02/M01/75/67/wKiom1Y4Xe2Ss64cAAD4eaPO6-8880.jpg "title=" Image 15.png "alt=" Wkiom1y4xe2ss64caad4eapo6-8880.jpg "/>

4.2 Download Loganalyzper and unzip to Web site root directory

Loganalyzper's official website is http://loganalyzer.adiscon.com/.

Here I chose the latest version of 4.1.3 to do the experiment.

wget tar-zxvf loganalyzer-4.1.3.tar.gzcp-r LOGANALYZER-4.1.3/SRC/VAR/WWW/HTML/LOGCP loganalyzer-4.1.3/contrib/*/ Var/www/html/logcd/var/www/html/logchmod +x configure.sh Secure.sh./configure.sh./secure.shchmod 666 config.php

4.3 Starting httpd and initializing the installation configuration

Service httpd Start

Then open the browser, access the http://10.134.140.66/log/install.php specific directory can be changed according to the entity situation.

650) this.width=650; "src=" http://s3.51cto.com/wyfs02/M01/75/65/wKioL1Y4YoKim_YwAALC91un_0g101.jpg "title=" Image 18.png "alt=" Wkiol1y4yokim_ywaalc91un_0g101.jpg "/>

Next, determine the configuration file

650) this.width=650; "src=" http://s3.51cto.com/wyfs02/M00/75/65/wKioL1Y4YumisuRLAAFelVszeY8216.jpg "title=" Image 21.png "alt=" Wkiol1y4yumisurlaafelvszey8216.jpg "/>

Next, configure log display information, etc.

650) this.width=650; "src=" Http://s3.51cto.com/wyfs02/M01/75/65/wKioL1Y4Y33zBTm0ABS3MPBoWTs044.bmp "title=" a1.bmp "alt=" Wkiol1y4y33zbtm0abs3mpbowts044.bmp "/>

Next, indicate back-end MySQL database connection information

650) this.width=650; "src=" Http://s3.51cto.com/wyfs02/M02/75/68/wKiom1Y4ZCKgO8pdAAIPFLcgwq0353.bmp "title=" a2.bmp "alt=" Wkiom1y4zckgo8pdaaipflcgwq0353.bmp "/>

Next, create the table in the back-end MySQL Database

650) this.width=650; "src=" Http://s3.51cto.com/wyfs02/M02/75/68/wKiom1Y4ZXWAvf_jABGayDPfRWg861.bmp "title=" a3.bmp "alt=" Wkiom1y4zxwavf_jabgaydpfrwg861.bmp "/>

Next, create an account password to manage Loganalyzper

650) this.width=650; "src=" Http://s3.51cto.com/wyfs02/M02/75/66/wKioL1Y4Zb-A7-GRABAyaK9BrIc385.bmp "title=" a4.bmp "alt=" Wkiol1y4zb-a7-grabayak9bric385.bmp "/>

Next, create the first test log message in the back-end MySQL Database

650) this.width=650; "src=" Http://s3.51cto.com/wyfs02/M00/75/66/wKioL1Y4Zl_wRUWhABb9YDFh2y8937.bmp "title=" a5.bmp "alt=" Wkiol1y4zl_wruwhabb9ydfh2y8937.bmp "/>

Next, Finish!

650) this.width=650; "src=" Http://s3.51cto.com/wyfs02/M02/75/66/wKioL1Y4ZkSwaLFwAA6hCOIJIYs265.bmp "title=" a6.bmp "alt=" Wkiol1y4zkswalfwaa6hcoijiys265.bmp "/>

4.4 Log it through the web!

650) this.width=650; "src=" http://s3.51cto.com/wyfs02/M01/75/68/wKiom1Y4Z7yxYE7GAATRkrPiN5Q708.jpg "title=" a7.png "alt=" Wkiom1y4z7yxye7gaatrkrpin5q708.jpg "/>

A few summary:

1. The database role in this architecture can actually be installed directly on the Rsyslog server, but for testing purposes, it is stripped

2. Rsyslog server can listen to the TCP514 can also UDP514, but usually recommended in UDP514, you understand;

3. Note: If nodes is indicated on the Rsyslog server collection, it is recommended that other unrelated lines in/etc/rsyslog.conf be commented out , that is, submitted to the remote server only. Of course, you can appropriately specify some high-level log information and then store a copy locally;

4. MySQL remote link to authorize, MySQL remote link to authorize, MySQL remote link to authorize , important words three times.
   

This article is from the "Blue Warehouse" blog, be sure to keep this source http://bluebox.blog.51cto.com/8852456/1709167

Rsyslog+mysql+loganalyzer Deploying a log server