Rsyslog from getting started to giving up

Server role is not to send and collect data, this data is divided into two categories, one is business data, mainly for the maintenance of business operations data, such as a game server, business data has equipment information, user level information, equipment explosion rate and so on. Another kind of data is the log information, this data is not necessary data, but this data can analyze user behavior, can further help us to find the user needs to create revenue source, in this big data age if not to make a point of data analysis basically dare not say that the host is a high-tech company. We also use log information to record server status, to help us understand the system bottlenecks, to achieve smooth business expansion contraction. In order to cater for the needs of the big data Age here is the main introduction to log collection.

Directory

• Log Collection

• Two Rsyslog features

• Three Rsyslog composition

• Four practices

Log Collection

The collection means of log data is varied, and the data collected is varied. Many of these applications collect data on their own, and this type of application is generally more heavyweight, and we also use its own Log collection tool, the main reason is that the data collected in a format is generally more special, need to record a lot of content. Logging requires writing data to disk, write operations typically consume a large amount of resources this requires a high-performance logging, if the use of third-party logging tools, every log is sent to the logging program, the intangible consumes a lot of resources. This kind of logging method needs to be combined with specific application to understand the use, here do not do too much introduction. Usually we will use some relatively small programs, these programs generate less logs but also more important to record, each small program to write a own logging tool is obviously more wasteful, then CentOS provides us with a common logging API, The applet log is called directly, the service name is Rsyslog, and as long as the program supports the use of Rsyslog logging, we can call it. Sometimes we collect very large amounts of data, which requires the use of higher performance Log collection display tools, the more popular is Elk,e:elasticsearch, L:logstash, K:kibana. I'll introduce you later .

Two Rsyslog features

• features of the Rsyslog: multithreading; Udp/tcp/ssl/tls/relp; store log information in MySQL, Pgsql, Oracle and other RDBMS; A powerful filter to filter the contents of any part of the log information; custom output format;

Common event Record Format: DateTime host Process [PID]: event content;

• Facility: Facilities, the data stream of the band is limited by a few; we are using LOCAL0-LOCAL7 custom data collection Auth, Authpriv, cron, daemon, Kern, LPR, mail, Mark, news, security, user, UUCP, syslog, Local0-local7

• proirity: Priority, in log is we select a level, as long as the error message exceeds this level will be recorded Debug, info, notice, warn (warning), err (Error), Crit (critical), Alert, Emerg (panic)

Three Rsyslog composition

Since is to understand Rsyslog first we want to understand his profile to facilitate me to understand its function, configuration file location in/etc/rsyslog.conf, its configuration file is divided into three parts

• MODULES

  This section is primarily for Rsyslog to launch some additional modules to help Rsyslog complete the collection of other host logs. $ModLoad imudp #启用udp监听模块, you can $UDPServerRun 514 #监听在514端口 $ModLoad imtcp$ without commenting it out Inputtcpserverrun 514$ModLoad ommysql #启用MySQL数据库记录数据, it's not enough. You need to define which database to call in the Rules section

• GLOBAL directives

  Global configuration segment, define common properties

• RULES

  defines the format of the logging, previously enabled in the Modules section of the MySQL module, where you can use the following format Facility.priority:ommysql:dbhost,dbname,dbuser,dbuserpassword

Four practices

First of all, we're going to implement a Rsyslog server that uses database logging information and help with another logging, and we provide a Web viewing interface. Split the task, 1 configuration rsyslog required database, 2 configuration Rsyslog write data to the database and listen to udp514 port, 3 use another host to send log information to the Rsyslog server, 4 provide Web viewing log tool Loganalyzer

Loganalyzer installation package download location http://loganalyzer.adiscon.com/downloads/

• Here I directly wrote a script, mainly for centos7,6 words need to change some of the content

• Yum Install Mariadb-server rsyslog-mysql-y #安装数据库和rsyslog连接数据库的组件

• Systemctl Start Mariadb.service #启动数据库

• MySQL <<eof GRANT all on syslog.* to ' rsluser ' @ ' 127.0.0.1 ' identified by ' rslpass ';

• GRANT all on syslog.* to ' rsluser ' @ ' localhost ' identified by ' rslpass ';

• EOF #授权rsyslog使用rsluser使用rslpass密码访问本地数据库, and can only be accessed locally

• MySQL </usr/share/doc/rsyslog-7.4.7/mysql-createdb.sql #创建rsyslog需要数据库, this SQL script is a rsyslog-mysql bag.

• systemctl Restart Rsyslog.service yum install httpd php php-mysql php-gd-y #安装提供网页组件

• Cd

• wget http://download.adiscon.com/loganalyzer/loganalyzer-4.1.5.tar.gz #下载loganalyzer包

• Tar XF loganalyzer-4.1.5.tar.gz #解压并复制到/var/www/html/loganalyzer

• cp-a Loganalyzer-4.1.5/src/var/www/html/loganalyzer CD

• Cd/var/www/html/loganalyzer #创建666权限的config. php

• Touch config.php

• chmod 666 config.php

• Systemctl Start Httpd.service #启动httpd服务

• Content that needs to be changed manually

• Vim/etc/rsyslog.conf

• $ModLoad IMUDP

• $UDPServerRun 514

• $ModLoad Ommysql #这一行需要添加

• *.info;mail.none;authpriv.none;cron.none:ommysql:127.0.0.1,syslog,rsluser,rslpass

• Configure the next server to write the log to this Rsyslog server, just change this line, of course, IP is your rsyslog server IP do not fill my IP

• *.info;mail.none;authpriv.none;cron.none @172.16.29.20

• Open Web page The seventh step of the configuration content is as follows

• my Syslog Source my SYSL og Source Source

• Typemysql Native Select

• Viewsyslogfields Database

• Type Options Table Typemonitorware Database

• Host127.0.0.1

• Database Namesyslog

• Database tablenamesystemevents

• Database Userrsluser

• Database Password ******** #这里的密码就是我们设置的连接数据库的密码rslpass Enable Row Counting No

• after the configuration is complete Change the permissions of the/var/www/html/loganalyzer/config.php file to 644

• chmod 644/var/www/html/loganalyzer/config.php

Summarize

This section is mainly to learn to use Rsyslog, mainly learn to configure the database to receive log information, let the Rsyslog listening port collects logs from other hosts, and configure the Web site to view the log information.

This article is from "Lao Wang Linux Journey" blog, please be sure to keep this source http://oldking.blog.51cto.com/10402759/1891232

Rsyslog from getting started to giving up