Use rsyslog + loganalyzer + mysql to deploy the log server in centos

As a system O & M engineer, I think it is a daily task to view and analyze LINUX system logs. However, after a long time, I find that every time I view the site logs, I have to go to the background one by one, several servers can do this, but if you manage hundreds of thousands of online servers, this method is too slow.

Later I thought that I could not have a log server to manage logs in a centralized manner, and showed the logs to the front-end for easy viewing in the form of WEB. The idea of code words suddenly came into being.

I have a habit of saving a memorandum of understanding for the LINUX system software that is recognized in the group or on the Internet, so that I can study it at leisure, some friends mentioned that rsyslog + loganalyzer was managing logs in a centralized manner some time ago, so today I just took the time to study the problem. Although there are twists and turns in the process, there are various pitfalls in the online documentation ), finally, we can use one day to share our understanding and build it for your reference only.

My new blog website has been created, and more new content will be updated on the new site soon ..

WelcomeHttp://www.showerlee.com

This document uses rsyslog + loganalyzer + mysql to centralize the system logs of all LINUX servers in the network to the log server for management. All logs are saved in the mysql database table.

Note: loganalyzer has two storage modes for obtaining client logs. One is to directly read the logs in the/var/log/directory of the client and save them to the directory on the server, one is read and saved to the log server database. The latter is recommended in this document.

Solution:

1. Environment deployment

Operating System: centos6.3 x64

Rsyslog: the default yum source.

Loganalyzer: loganalyzer-3.6.3

LAMP: httpd-2.4.4, mysql-5.6.10, php-5.4.13

650) this. width = 650; "src =" http://www.bkjia.com/uploads/allimg/131227/1JTA555-0.jpg "title =" rsyslog.jpg "/>

Rsyslog server: 192.168.7.201 lamp.example.com

Rsyslog client: 192.168.7.74 www2.example.com

1. Install the LAMP Environment

Ben Bo Portal: http://showerlee.blog.51cto.com/2047005/1174141

2. Disable iptables and SELINUX.

# Service iptables stop

Note: To enable the iptables service to increase system security

The server needs to add the rsyslog UDP 514 port and the loganalyzer TCP 80 port passing rule.

# Iptables-a input-p udp -- dport 514-j ACCEPT

# Iptables-P OUTPUT ACCEPT

# Iptables-a input-p TCP -- dport 80-j ACCEPT

Clients only need to add OUTPUT rules

# Iptables-P OUTPUT ACCEPT

The rules show that the rsyslog server passively acquires data and the client actively sends data.

If you disable iptables, you can ignore it ..

# Setenforce 0

# Vi/etc/sysconfig/selinux

---------------

SELINUX = disabled

---------------

3. Synchronization time

# Ntpdate asia.pool.ntp.org

Ii. install and configure rsyslog

(Rsyslog server)

# Yum install rsyslog-mysql-y

Note: rsyslog-mysql transmits logs to a module of the mysql database for rsyslog. installation is required here.

# Cd/usr/share/doc/rsyslog-mysql-5.8.10/

# Mysql-uroot-p123456 <createDB. SQL

Note: here, the import database operation is finally studied by the real-time blogger, that is, the Syslog database is created and two empty tables are created in the database.

Create rsyslog user permissions in mysql

# Mysql-uroot-p123456

> Grant all privileges on Syslog. * to rsyslog @ localhost identified by "123456 ";

> Flush privileges;

> Exit

Configure the server to support the rsyslog-mysql module, and enable the UDP Service port to obtain logs of other LINUX systems in the network.

# Vi/etc/rsyslog. conf

Add the two lines under #### MODULES ###

------------------

$ ModLoad ommysql. so

*. *: Ommysql: localhost, Syslog, rsyslog, 123456

------------------

Note: localhost indicates the local host, Syslog indicates the database name, rsyslog indicates the database user, and 123456 indicates the user password.

Uncomment the following three lines

-----------------

$ ModLoad immark

$ ModLoad imudp

$ UDPServerRun 514

-----------------

Restart the service:

# Service rsyslog restart

(Rsyslog client)

# Yum install rsyslog-y

Configure the rsyslog client to send local logs to the server

# Vi/etc/rsyslog. conf

Add the following content to the last line:

-------------------

*. * @ 192.168.7.201

-------------------

Note: 192.168.7.201 is the IP address of the log server.

Restart the service:

# Service rsyslog restart

3. Install loganalyzer

# Wget http://download.adiscon.com/loganalyzer/loganalyzer-3.6.3.tar.gz

# Tar zxvf loganalyzer-3.6.3.tar.gz

# Cd loganalyzer-3.6.3

# Mkdir-p/usr/local/apache2/htdocs/loganalyzer

Copy the loganalyzer source code to the loganalyzer directory under the apache DocumentRoot.

# Cp-r src/*/usr/local/apache2/htdocs/loganalyzer

# Cp-r contrib/*/usr/local/apache2/htdocs/loganalyzer

# Chown-R daemon. daemon/usr/local/apache2/htdocs/loganalyzer

Before installing loganalyzer through the web wizard, you must first execute the following two scripts

# Cd/usr/local/apache2/htdocs/loganalyzer/

# Sh configure. sh

# Sh secure. sh

Note: This script is used to create config. php under the Directory and configure the file permissions.

Enter the URL in the browser to enter the Installation Wizard

Http: // 192.168.7.201/loganalyzer

1. If no configuration file is displayed, click here to generate it using the wizard

650) this. width = 650; "src =" http://www.bkjia.com/uploads/allimg/131227/1JTA344-1.png "style =" float: none; "title =" 1.png"/>

2. NEXT

650) this. width = 650; "src =" http://www.bkjia.com/uploads/allimg/131227/1JT64408-2.png "style =" float: none; "title =" 2.png"/>

3. Follow the input configuration and click NEXT:

650) this. width = 650; "src =" http://www.bkjia.com/uploads/allimg/131227/1JT64296-3.png "title =" 14.png"/>

Note: if an error is reported when you click NEXT, run the following command in the background to continue

# Ln-s/var/lib/mysql. sock/tmp/mysql. sock

4. Start writing data to the database. NEXT

650) this. width = 650; "src =" http://www.bkjia.com/uploads/allimg/131227/1JTC452-4.png "title =" 5.png"/>

5. Prompt that the write is successful. NEXT

650) this. width = 650; "src =" http://www.bkjia.com/uploads/allimg/131227/1JTCc0-5.png "title =" 6.png"/>

6. Set the administrator account. After the configuration is completed, click NEXT.

650) this. width = 650; "src =" http://www.bkjia.com/uploads/allimg/131227/1JT62A2-6.png "title =" 7.png"/>

7. Set monitoring logs to be saved to the mysql database, and click

650) this. width = 650; "src =" http://www.bkjia.com/uploads/allimg/131227/1JT62934-7.png "title =" 15.png"/>

650) this. width = 650; "src =" http://www.bkjia.com/uploads/allimg/131227/1JT635F-8.png "title =" 16.png"/>

8. Complete the configuration and FINISH

650) this. width = 650; "src =" http://www.bkjia.com/uploads/allimg/131227/1JT62133-9.png "title =" 10.png"/>

9. Enter the logon page:

650) this. width = 650; "src =" http://www.bkjia.com/uploads/allimg/131227/1JT64V2-10.png "title =" 11.png"/>

10. Go to the main interface:

Check whether loganalyzer obtains system logs of 192.168.7.201 and 192.168.7.74.

650) this. width = 650; "src =" http://www.bkjia.com/uploads/allimg/131227/1JT64612-11.png "title =" 19.png"/>

Use navicat to check whether the rsyslog server and client system logs are written to the database Syslog-SystemEvents table.

650) this. width = 650; "src =" http://www.bkjia.com/uploads/allimg/131227/1JT64963-12.png "title =" 18.png"/>

---------- Success ------------

Postscript:

1. I have configured a web Background account in the loganalyzer Installation wizard. I cannot log on to the logon page, and I am prompted that the account or password is incorrect. Why? Finally, the login authentication was barely passed only after being removed from the wizard. If you encounter the same problem, I hope to help you solve it.

After a friend gave me some advice, I finally found the reason.

First, use the following command

# Tail-f/usr/local/mysql/log/mysql. log

In step 6 of the loganalyzer wizard, enter the web Background account password and click NEXT

An INSERT statement appears in the mysql. log.

650) this. width = 650; "src =" http://www.bkjia.com/uploads/allimg/131227/1JTA951-13.png "title =" Your png"/>

Then copy the statement to the background and execute it manually to check the error message.

# Mysql-uroot-p123456;

> Insert into logcon_users (username, password, is_admin) VALUES ('admin', '00a1f187721cxxxxxxx6bf791e69382c ', 1 );

ERROR 1364 (HY000): Field 'last _ login 'doesn' t have a default value

The prompt 'last _ login' cannot be null.

OK. log on to navicat and set this column to allow null values to be saved.

650) this. width = 650; "src =" http://www.bkjia.com/uploads/allimg/131227/1JT61017-14.png "title =" 21.png"/>

Execute this statement again

> Insert into logcon_users (username, password, is_admin) VALUES ('admin', '00a1f187721cxxxxxxx6bf791e69382c ', 1 );

Query OK, 1 row affected (0.06 sec)

Show execution successful

View the table in the background using navicat. A record is successfully inserted.

650) this. width = 650; "src =" http://www.bkjia.com/uploads/allimg/131227/1JT61495-15.png "title =" 22.png"/>

You can log on to the loganalyzer web Background with this account again.

It seems that when mysql runs this statement, it finds that the last_login column is not empty by default. Therefore, it rejects the execution of this insert statement. The solution is to set this column to allow null values.

The old version of the mysql-5.0.56 online without manual operation, you can successfully log on to the background

But the problem with this article is that in the mysql-5.6.10 version

It seems that this is not a BUG in loganalyzer. It should be because mysql has improved the rigor of executing the insert statement in a later version.

Here, I will give my friends some ideas here for their reference only.

I would like to thank longeleven11 for your mention ....

2. the process of installing rsyslog is tortuous. Before that, I wanted to compile and install all the documents on the Internet. It took two hours to compile rsyslog to install six dependency packages, all the documents from foreigners are tested one by one. After installing the documents provided on the Internet, they cannot reasonably provide how to coexist with the rsyslog of the system without conflict, that is, how does the compiled rsyslog startup method differ from the built-in system? Simply use yum, Which is helpless. There are also various intermittent attacks by google. As for the reason, everyone knows...

3. Install Windows client (win2008 server 64bit)

1. Download evtsys

Http://eventlog-to-syslog.googlecode.com/files/Evtsys_4.4.3_64-Bit.zip

2. decompress the package and copy all the files in the 64-Bit folder to C: \ Windows \ System32.

3. Enable the evtsys Service

Run-cmd

> Cd c: \ Windows \ System32

> Evtsys-I-s 10-h 192.168.7.11-p 514

> Net start evtsys

4. Verify the effect

650) this. width = 650; "src =" http://www.bkjia.com/uploads/allimg/131227/1JT625F-16.png "title =" 1.png"/>

This article from "all the way to the North" blog, please be sure to keep this source http://showerlee.blog.51cto.com/2047005/1231160