CENTOS6 system Rsyslog+loganalyzer+mysql Deployment Log Server

Introduction of Environment Configuration

1. Introduction of experimental environment

Rsyslog Server Os:centos Release 6.5 (Final) x86_64
Rsyslog SERAVR ip:211.149.204.229
Rsyslog Client Os:centos Release 6.5 (Final) x86_64
Rsyslog Client ip:63.223.64.110
Rsyslog version:rsyslog-5.8.10-8.el6.x86_64
LNMP Version:nginx 1.6 +mysql 5.1.72 + PHP 5.3.28

2. Dependent Package Installation

[root@ebs-28723 src]# yum install gcc gcc-c++ make Rsyslog rsyslog-mysql unzip
3, Configuration Tips
3.1, because the PHP environment is installed by default, this article does not describe if the configuration and installation of PHP environment, you can configure the LNMP or lamp, the process of adding a site does not describe
3.2, PHP Environment requirements:
php-mysql:php connection to Mysql extension
Extension of php-gd:php Drawing
4. Experimental purposes
Facilitate centralized management of server logs

Ii. introduction of Rsyslog

1. Basic Introduction

Rsyslog is a syslogd multi-threaded enhanced version, Fedora and Ubuntu,rhel default log systems are all rsyslog. Rsyslog is responsible for logging, logrotate for backing up and deleting old logs, and updating log files
2. Advantages
2.1, the unification of the log, centralized management
2.2, real-time log transmission to a more secure remote server, the real record of user behavior, so that the log 2 changes in the probability of greatly reduced, so that the log can be true playback, easy to track the problem.
3, the new function of Rsyslog:
Rsyslog is a enhanced version of the Syslog, with a variety of pro functions, typically:
3.1. Write the log directly to the database.
3.2. Journal queues (memory queues and disk queues)
3.3, flexible template mechanism, you can get a variety of output formats.
3.4, plug-in structure, a wide range of input, output module
3.5, you can store the log in the mysql,postgresql,oracle and other databases
4, Rsyslog Package Introduction
4.1. Installation software

[root@ebs-28723 ~]# Rpm-qa | grep syslog
Rsyslog-5.8.10-8.el6.x86_64//rsyslog Package
rsyslog-mysql-5.8.10-8.el6.x86_64//rsyslog-mysql Software Installation package
4.2. Introduction of software Files

[root@ebs-28723 ~]# rpm-ql Rsyslog
/etc/logrotate.d/syslog
/etc/pki/rsyslog
/etc/rc.d/init.d/rsyslog
/etc/rsyslog.conf//rsyslog configuration file
/etc/rsyslog.d
/etc/sysconfig/rsyslog
/lib64/rsyslog
/lib64/rsyslog/imfile.so//rsyslog module, I start with input module
/lib64/rsyslog/imklog.so
/lib64/rsyslog/immark.so
/lib64/rsyslog/impstats.so
/lib64/rsyslog/imptcp.so
/lib64/rsyslog/imtcp.so
/lib64/rsyslog/imudp.so
/lib64/rsyslog/imuxsock.so
/lib64/rsyslog/lmnet.so
/lib64/rsyslog/lmnetstrms.so
/lib64/rsyslog/lmnsd_ptcp.so
/lib64/rsyslog/lmregexp.so
/lib64/rsyslog/lmstrmsrv.so
/lib64/rsyslog/lmtcpclt.so
/lib64/rsyslog/lmtcpsrv.so
/lib64/rsyslog/lmzlibw.so
/lib64/rsyslog/ommail.so
/lib64/rsyslog/omprog.so
/lib64/rsyslog/omruleset.so
/lib64/rsyslog/omtesting.so
/lib64/rsyslog/omuxsock.so
/lib64/rsyslog/pmlastmsg.so
4.3, rsyslog configuration file Introduction (/etc/rsyslog.conf)

# # # # # # # # # MODULES
$ModLoad Imuxsock # Imuxsock is a module name that supports the Local System log module
$ModLoad Imklog # Imklog is a module name that supports kernel log modules
# $ModLoad Immark # Immark is a module name that supports log marks
# provides UDP syslog reception
# $ModLoad IMUDP # imdup is a module name that supports the UDP protocol
# $UDPServerRun 514 # Allow 514 ports to receive logs forwarded over UDP and TCP protocols
# provides TCP syslog reception
# $ModLoad Imtcp # imtcp is a module name that supports the TCP protocol
# $InputTCPServerRun 514
# GLOBAL Directives #
# Use default timestamp format
$ActionFileDefaultTemplate Rsyslog_traditionalfileformat #定义日志格式默认模板
# File syncing capability is disabled by default. This feature was usually not required,
# not useful and a extreme performance hit
# $ActionFileEnableSync on
# Include all config files in/etc/rsyslog.d/
$IncludeConfig/etc/rsyslog.d/*.conf All files in conf #载入rsyslog. d files
# The RULES # #
# Log all kernel messages to the console.
# Logging Much else clutters up the screen.
#kern. */dev/console
# Log anything (except mail) of level info or higher.
# Don ' t log private authentication messages!
*.info;mail.none;authpriv.none;cron.none/var/log/messages
#记录所有日志类型的info级别以及大于info级别的信息到/var/log/mess, except for mail message information, AUTHPRIV authentication information and cron time task-related information
# The Authpriv file has restricted access.
authpriv.*/var/log/secure
#authpriv验证相关的所有信息存放在/var/log/secure
# Log All of the mail messages in one place.
Mail.*-/var/log/maillog
#邮件的所有信息存放在/var/log/maillog
# Log Cron Stuff
Cron.*/var/log/cron
#计划任务有关的信息存放在/var/log/cron
# Everybody gets emergency messages
*.emerg *
#记录大于等于emerg级别信息, sent to each person logged on to the system in a wall manner
# Save News errors of level crit and higher in a special file.
Uucp,news.crit/var/log/spooler
#记录uucp, News,crit etc stored in/var/log/spooler
# Save Boot messages also to Boot.log
local7.*/var/log/boot.log
#本地服务的启动的所有日志存入在/var/log/boot.log.
# ### begin forwarding Rule ###
# The statement between the begin ... define a single forwarding
# rule. They belong Together, does not split them. If you create multiple
# forwarding rules, duplicate the whole block!
# Remote Logging (we use TCP for reliable delivery)
#
# an On-disk the queue is created to this action. If the remote host is
# down, messages are spooled to disk and sent the it is up again.
# $WorkDirectory/var/lib/rsyslog # Where to place spool files
# $ActionQueueFileName FwdRule1 # unique name prefix for spool files
# $ActionQueueMaxDiskSpace 1g # 1GB space limit (use as much as possible)
# $ActionQueueSaveOnShutdown on # Save messages to disk on shutdown
# $ActionQueueType LinkedList # Run asynchronously
# $ActionResumeRetryCount-1 # Infinite retries if host is down
# remote host is:name/ip:port, e.g. 192.168.0.1:514, port Optional
#*.* @ @remote-host:514
# ### end of the forwarding rule ###
4.4, the rsyslog.conf of the definition of the log Rules format
Definition format for log rules in #rsyslog. conf

Facitlity.priority Target
#facility: Log device (can be understood as log type)
======================================================
Auth # pam generates logs, authentication logs
Authpriv # ssh,ftp and other login information verification information, certification Authority certification
Cron # Time Task related
Kern # Kernel
LPR # Print
Mail # Messages
Mark (syslog) # Rsyslog Service internal information, time identification
News # Newsgroups
User # The information generated by the program
UUCP # UNIX to UNIX Copy,unix hosts related communications
Local 1~7 # custom log device
======================================================
#priority: Log Level
======================================================
Debug # with debug information, maximum log information
Info # General Information log, most commonly used
Notice # Information of the most important general conditions
Warning,warn # Warning Level
Err,error # Error level, information that prevents a feature or module from working correctly
Crit #严重级别 to prevent a system or entire software from working correctly
Alert #需要立即修改的信息
Emerg,panic #内核崩溃等严重信息
#从上而下, levels from low to high, record less information, and if you set the level of the log to err, the log will not record less than the Err level

5, the introduction of logroate log scrolling

5.1. Basic Introduction
All log files grow rapidly over time and with increasing number of visits, so regular cleanup of log files is necessary to avoid unnecessary waste of space for rent reduction, as well as the time it takes for administrators to view logs. As a result, there is a need for logrotate, Redhat system has been installed by default Logrotate and the use of logrotate set up the Rsyslog log of the rapid growth of the settings.
The execution of Logrotate is implemented by the Crond service. In the/etc/cron.daily directory, there is a file logrotate, which is actually a shell script that the user launches Logroate
5.2, installation package introduction

[root@ebs-28723 ~]# Rpm-qa logrotate
Logrotate-3.7.8-17.el6.x86_64 #logrotate软件安装的版本
[root@ebs-28723 ~]# rpm-ql logrotate
/etc/cron.daily/logrotate
/etc/logrotate.conf #logrotate的配置文件
/ETC/LOGROTATE.D #logrotate的子目录
/usr/sbin/logrotate
/usr/share/doc/logrotate-3.7.8
/usr/share/doc/logrotate-3.7.8/changes
/usr/share/doc/logrotate-3.7.8/copying
/usr/share/man/man5/logrotate.conf.5.gz
/usr/share/man/man8/logrotate.8.gz
/var/lib/logrotate.status

5.3 Logrotate Configuration files (/etc/logrotate.conf and/etc/logrotate.d/syslog)

#/etc/logrotate.conf
# "Man logrotate" for details
# Rotate log Files Weekly
Weekly # Clean up log files once a week
# Keep 4 weeks worth of backlogs
Rotate 4 # Keep four rotation logs
# Create new (empty) log files after the rotating old ones
Create # Clean up old logs while creating new blank logs
# Use date as a suffix of the rotated file
Dateext # Use a rollback file with a date suffix
# Uncomment this if you want your log files compressed
#compress # compression
# RPM Packages Drop log rotation information into this directory
INCLUDE/ETC/LOGROTATE.D #包含了 The configuration file in the/etc/logrotate.d/directory
# no packages own wtmp and btmp--we'll rotate them here
/var/log/wtmp {# to/var/log/wtmp This log file is rolled back according to the following setting log
Monthly # Rotate once a month
Create 0664 Root utmp # set wtmp This log file permissions, owner, group
MinSize 1M # Log file must be greater than 1M to rotate (rollback)
Rotate 1 # Save a rotation log
}
/var/log/btmp {
Missingok #如果文件丢失不报错
Monthly
Create 0600 Root utmp
Rotate 1
}
# System-specific logs may also is configured here.
#/etc/logrotate.d/syslog
/var/log/cron the path of target specified by the global configuration definition in the #这些文件是rsyslog. conf file
/var/log/maillog
/var/log/messages
/var/log/secure
/var/log/spooler
{
Sharedscripts
Postrotate #轮换之后重启rsyslog服务
/bin/kill-hup ' cat/var/run/syslogd.pid 2>/dev/null ' 2>/dev/null | | True
Endscript
}

Third, Rsyslog configuration

1. Rsyslog Server Configuration
1.1, Rsyslog Software Installation

[root@ebs-28723 ~]# Yum Install rsyslog-mysql-y
#将日志传送到MySQL数据库的一个模块, this must be installed
1.2, MySQL Configuration
#导入数据库

[root@ebs-28723 ~]# Mysql-uroot-p </usr/share/doc/rsyslog-mysql-5.8.10/createdb.sql
#创建用户

Mysql> grant all privileges in syslog.* to syslog@localhost identified by ' syslog_123 ';
Mysql> grant all privileges in syslog.* to syslog@ '% ' identified by ' syslog_123 ';
mysql> flush Privileges;
1.3, configure the server to support the Rsyslog-mysql module, and open the UDP service port to obtain other Linux system log in the network
Modify/etc/rsyslog.conf, as follows
Hint: The original file should be backed up before modification
#注释掉以下行

*.info;mail.none;authpriv.none;cron.none
authpriv.*
Mail.*
cron.*
*.emerg
#在注释掉的第一行下面添加

$ModLoad Ommysql
*.*: ommysql:localhost,syslog,syslog,syslog_123 #格式为: ommysql:mysqlhost,mysqlname,mysqluser,mysqluserpasswd
#开启以下模块 (Cancel the previous note)

$ModLoad Imuxsock
$ModLoad IMUDP
$UDPServerRun 514
and restart the Rsyslog service.

Service Rsyslog Restart

1.4, iptables Configuration

[root@ebs-28723 ~]# iptables-a input-p UDP--dport 514-j
[root@ebs-28723 ~]# iptables-a output-p UDP--sport 514-j
[root@ebs-28723 ~]# Service iptables restart
2, Rsyslog Client Configuration
2.1. Edit the Rsyslog configuration file to upload to the Rsyslog server machine
Hint: The original file should be backed up before modification
#注释掉以下行

*.info;mail.none;authpriv.none;cron.none
authpriv.*
Mail.*
cron.*
*.emerg
#在注释掉的第一行下面添加

IP of the *.* @211.149.204.229 #IP为rsyslog server
and restart the Rsyslog service.

Service Rsyslog Restart
2.2, iptables Configuration

[Root@web_us ~]# iptables-a input-p UDP--dport 514-j
[Root@web_us ~]# iptables-a output-p UDP--sport 514-j
[Root@ewb_us ~]# Service iptables restart
Iv. Introduction and configuration of Loganalyzer

1, Loganlyzer Introduction
Official website: http://loganalyzer.adiscon.com/
Download Address: http://down.shshenchu.com/loganalyzer-3.6.5.tar.gz
Chinese Language pack: Http://down.shshenchu.com/loganalyzer3.6.5-lan-zh.zip
is a PHP language written in the log Analysis tool, can easily simplify the view of the log
2. Configure Loganlyzer
2.1, download the file

[root@ebs-28723 ~]# CD/USR/LOCAL/SRC
[root@ebs-28723 src]# wget http://down.shshenchu.com/loganalyzer-3.6.5.tar.gz
[root@ebs-28723 src]# wget Http://down.shshenchu.com/loganalyzer3.6.5-lan-zh.zip

2.2, File decompression

[root@ebs-28723 src]# TAR-ZXVF loganalyzer-3.6.5.tar.gz
[root@ebs-28723 src]# Unzip Loganalyzer3.6.5-lan-zh.zip

2.3, move to the site Directory

[root@ebs-28723 src]# cp-r loganalyzer-3.6.5/src/*/web/log/public_html/
[root@ebs-28723 src]# CP loganalyzer-3.6.5/contrib/*.sh/web/log/public_html/
[root@ebs-28723 src]# cp-r LOGANALYZER3.6.5-LAN-ZH/WEB/LOG/PUBLIC_HTML/LANG/CN

2.4. Create MySQL Database

#创建用户
[root@ebs-28723 ~]# Mysql-uroot-p
mysql> CREATE DATABASE Loganalyzer;
Mysql> grant all privileges in lyzeruser.* to lyzeruser@localhost identified by ' lyzeruser_123 ';
mysql> flush Privileges;
2.5. Installation visit

2.6. Modify language
It is generally recommended that Victor use English, if need to change for Chinese words, a little trouble, the following process
1. Admin Center ====>default character encoding (selected as UTF8) #修改语言的默认编码
2. Admin Center ====>default selected language (selected as Chinese) #修改默认的语言显示
Then click on the upper right corner to select Chiense for the currently displayed language in select language!