Build a concentrated log server using syslog-ng

Rhel5 and centos 5.5 x86_64 are all tested.

In the production environment, there is a log server dedicated to recording the log information of other servers is a good idea, but with the Red Hat built-in syslog, the configuration is simple, however, there is no way to separate logs. By default, logs are all heap in the/var/log/message file, which is used to create a log server. The following describes how to use syslog-ng to build a log server, this also supports importing logs to the database and publishing logs through the web page. It sounds quite powerful. Next, let's take a good look ......

Environment Introduction
Log Server IP Address: 192.168.90.20; Client IP Address: 192.168.90.10
System: rhel5.4
Purpose: automatically store client logs in the corresponding directory on the server, and save the logs separately Based on the date, IP address, and log type.
Note: Due to operations in the virtual machine environment, the time of the server on the client is not synchronized, so there will be inconsistent log recording time;

[Root @ server2 ~] # Mkdir-P/Usr/local/src/tarbag/
[Root @ server2 ~] # Cd/usr/local/src/tarbag/
[Root @ server2 tarbag] # wget http://www.balabit.com/downloads/files/eventlog/0.2/eventlog_0.2.9.tar.gz
[Root @ server2 tarbag] # tar-zxvf eventlog_0.2.9.tar.gz-C ../software/
[Root @ server2 tarbag] # CD ../software/eventlog-0.2.9/
[Root @ server2 eventlog-0.2.9] #./configure -- prefix =/usr/local/EventLog & make install
[Root @ server2 eventlog-0.2.9] # ls/usr/local/EventLog/
Include lib

[Root @ server2 syslog-ng-3.0.5] # cd-
/Usr/local/src/tarbag
[Root @ server2 tarbag] # wgethttp: // www.balabit.com/downloads/files/libol/0.3/libol-0.3.9.tar.gz
[Root @ server2 tarbag] # tar-zxvf libol-0.3.9.tar.gz-C ../software/
[Root @ server2 tarbag] # CD ../software/libol-0.3.9/
[Root @ server2 libol-0.3.9] #./configure -- prefix =/usr/local/libol & make install
[Root @ server2 libol-0.3.9] # ls/usr/local/libol/
Bin include lib

[Root @ server2 tarbag] # wgethttp: // www.balabit.com/downloads/files/syslog-ng/sources/3.0.5/source/syslog-ng_3.0.5.tar.gz
[Root @ server2 tarbag] # tar-zxvf syslog-ng_3.0.5.tar.gz-C ../software/
[Root @ server2 tarbag] # CD ../software/syslog-ng-3.0.5/
[Root @ server2 syslog-ng-3.0.5] # exportpkg_config_path =/usr/local/EventLog/lib/pkgconfig
[Root @ server2 syslog-ng-3.0.5] #./configure -- prefix =/usr/local/syslog-ng -- With-libol =/usr/local/libol & make install
Configure: Error: cannot find EventLog version >=0.2: Is PKG-config inpath? (If this error occurs, it is basically because the previous pkg_config_path variable is not specified)
[Root @ server2 syslog-ng-3.0.5] # ls/usr/local/syslog-ng/
Bin libexec sbin share
[Root @ server2 syslog-ng-3.0.5] # mkdir/usr/local/syslog-ng/etc
[Root @ server2 syslog-ng-3.0.5] # mkdir/usr/local/syslog-ng/var
[Root @ server2 syslog-ng-3.0.5] # cp contrib/syslog-ng.conf.RedHat/usr/local/syslog-ng/etc/
[Root @ server2 syslog-ng-3.0.5] # cp contrib/init. d. RedHat/etc/init. d/syslog-ng

[Root @ server2 syslog-ng-3.0.5] # cd/usr/local/syslog-ng/etc/
[Root @ server2 etc] # Music syslog-ng.conf.RedHat syslog-ng.conf

[Root @ server2 etc] # Cat syslog-ng.conf # copy the following configuration

[Root @ server2 etc]

@ Version: 3.0
Options {
Long_hostnames (off );
Log_msg_size (8192 );
Flush_lines (1 );
Log_1_o_size (20480 );
Time_reopen (10 );
Use_dns (yes );
Dns_cache (yes );
Use_fqdn (yes );
Keep_hostname (yes );
Chain_hostnames (NO );
Perm (0644 );
Stats_freq (43200 );
};
Source s_internal {internal ();};
Destination d_syslognglog {file ("/var/log/syslog-ng.log ");};
Log {source (s_internal); destination (d_syslognglog );};

Source s_local {
Unix-dgram ("/dev/log ");
File ("/proc/kmsg" program_override ("kernel :"));
};

Filter f_messages {level (info .. emerg) ;};# define 7 log types
Filter f_secure {facility (authpriv );};
Filter f_mail {facility (Mail );};
Filter f_cron {facility (cron );};
Filter f_emerg {level (emerg );};
Filter f_spooler {level (CRIT .. emerg) and facility (uucp, News );};
Filter f_local7 {facility (local7 );};
Destination d_messages {file ("/var/log/messages") ;};# define the location of seven types of logs on the client
Destination d_secure {file ("/var/log/secure ");};
Destination d_maillog {file ("/var/log/maillog ");};
Destination d_cron {file ("/var/log/cron ");};
Destination d_console {usertty ("root ");};
Destination d_spooler {file ("/var/log/Spooler ");};
Destination d_bootlog {file ("/var/log/dmesg ");};
Log {source (s_local); filter (f_emerg); destination (d_console );};
Log {source (s_local); filter (f_secure); destination (d_secure); flags (final );};
Log {source (s_local); filter (f_mail); destination (d_maillog); flags (final );};
Log {source (s_local); filter (f_cron); destination (d_cron); flags (final );};
Log {source (s_local); filter (f_spooler); destination (d_spooler );};
Log {source (s_local); filter (f_local7); destination (d_bootlog );};
Log {source (s_local); filter (f_messages); destination (d_messages );};

# Remote logging # define the listening port
Source s_remote {
TCP/IP (IP (0.0.0.0) Port (514 ));
UDP (IP (0.0.0.0) Port (514 ));
};
# Define the format, location, and permissions of client logs stored on the server
Destination r_console {file ("/var/log/syslog-ng/$ year $ month $ day/$ host/console" Owner ("root") group ("root ") perm (0640) dir_perm (0750) create_dirs (yes ));};
Destination r_secure {file ("/var/log/syslog-ng/$ year $ month $ day/$ host/secure" Owner ("root") group ("root ") perm (0640) dir_perm (0750) create_dirs (yes ));};
Destination r_cron {file ("/var/log/syslog-ng/$ year $ month $ day/$ host/cron" Owner ("root") group ("root ") perm (0640) dir_perm (0750) create_dirs (yes ));};
Destination r_spooler {file ("/var/log/syslog-ng/$ year $ month $ day/$ host/Spooler" Owner ("root") group ("root ") perm (0640) dir_perm (0750) create_dirs (yes ));};
Destination r_bootlog {file ("/var/log/syslog-ng/$ year $ month $ day/$ host/bootlog" Owner ("root") group ("root ") perm (0640) dir_perm (0750) create_dirs (yes ));};
Destination r_messages {file ("/var/log/syslog-ng/$ year $ month $ day/$ host/messages" Owner ("root") group ("root ") perm (0640) dir_perm (0750) create_dirs (yes ));};
Log {source (s_remote); filter (f_emerg); destination (r_console );};
Log {source (s_remote); filter (f_secure); destination (r_secure); flags (final );};
Log {source (s_remote); filter (f_cron); destination (r_cron); flags (final );};
Log {source (s_remote); filter (f_spooler); destination (r_spooler );};
Log {source (s_remote); filter (f_local7); destination (r_bootlog );};
Log {source (s_remote); filter (f_messages); destination (r_messages );};

[Root @ server2 etc] # chmod + x/etc/init. d/syslog-ng
[Root @ server2 etc] # chkconfig -- add syslog-ng
Service syslog-ng does not support chkconfig (if this error occurs, modify the first four lines of the script as follows)
[Root @ server2 etc] # Head-4/etc/init. d/syslog-ng// Add services not to the etc in USR
#! /Bin/bash
# Chkconifg: -- add syslog-ng
# Chkconfig: 2345 12 88
# Description: syslog-ng

The script also needs to modify the following three locations
[Root @ server2 etc] # grep 'path'/etc/init. d/syslog-ng
Path =/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/syslog-ng/bin:/usr/local/syslog-ng/sbin
[Root @ server2 etc] # grep 'init '/etc/init. d/syslog-ng | head-2
Init_prog = "/usr/local/syslog-ng/sbin/syslog-ng" # full path to daemon
Init_opts = "-F/usr/local/syslog-ng/etc/syslog-ng.conf" # options passed to daemon

[Root @ server2 etc] # service syslog-ng start// Pay attention to CD/usr/local/syslog-ng/etc/
Starting syslog-NG:/usr/local/syslog-ng/sbin/syslog-NG: Error while loading shared libraries: libevtlog. so.0: cannot open shared object file: no such file or directory
Starting kernel logger: this error occurs because the Shared Library link is not properly configured.
[Root @ server2 etc] # ln-S/usr/local/EventLog/lib/*/lib/

The following error occurs because the main configuration file is missing: @ version: 3.0.
Starting syslog-NG: configuration file has no version number, assumingsyslog-ng 2.1 format. Please add @ version: maj. min to the beginning of thefile;

[Root @ server2 ~] # Service syslog-ng start
Starting kernellogger: [OK]
[Root @ server2 etc] # Cat/var/log/syslog-ng.log
Oct 15 11:29:47 localhost. localdomain syslog-NG [28022]: syslog-ng starting up; version = '3. 123'
[Root @ server2 etc] # netstat-tpln | grep 514
TCP 0 0 0.0.0.0: 514 0.0.0.0: * Listen 28407/syslog-ng

[Root @ server2 etc] # ps aux | grep Syslog
Root 2385 0.0 0.1 5908 608? SS syslogd-M 0
Root 28406 0.0 0.1 49176 596? S supervising syslog-ng
Root 28407 0.0 0.4 53468 2100? SS/usr/local/syslog-ng/sbin/syslog-ng-F/usr/local/syslog-ng/etc/syslog-ng.conf
Root 28417 0.0 0.1 61156 720 pts/3 R + grep Syslog

Enabled successfully

Client Configuration:
[Root @ client ~] # Tail-1/etc/syslog. conf
*. * @ 192.168.90.20
[Root @ client ~] # Service syslog restart
[Root @ client ~] # Logger-I just one test
[Root @ client ~] # Tail-1/var/log/messages
Oct 11:38:45 localhost kernel: klogd 1.4.1, log source =/proc/kmsg started.
[Root @ server2 ~] # Cat/var/log/syslog-ng/20121015/192 .168.90.10/messages
Oct 15 11:38:50 192.168.12.138 root [24984]: Just one test

Reference: http://blog.sina.com.cn/s/blog_4a071ed80100cssu.html

Syslog-ng has been configured. The following briefly describes how to save system logs to MySQL.
1: link the MySQL header file and library file to/usr/local
[Root @ server2 ~] # Ln-S/usr/local/MySQL/lib/MySQL/usr/local/lib/MySQL
[Root @ server2 ~] # Ln-S/usr/local/MySQL/include/MySQL // usr/local/include
[Root @ server2 ~] # Cd/usr/local/src/software/sqlsyslogd
2: Download The sqlsyslogd source code package. This is because the entire directory downloads the index file with the index.html header.
[Root @ server2 software] # wget-d-r-NP
Http://www.frasunek.com/sources/security/sqlsyslogd/
[Root @ server2 software] # cd
Www.frasunek.com/sources/security/sqlsyslogd/
[Root @ server2 sqlsyslogd] # rm-RF index.html *
[Root @ server2 sqlsyslogd] # cd contrib/
[Root @ server2 contrib] # rm-RF index.html *
[Root @ server2 contrib] # cd
[Root @ server2 ~] # Mv/usr/local/src/software/www.frasunek.com/sources/security/sqlsyslogd// usr/
Local/src/software/
3: Make. Copy the sqlsyslogd binary program to the/usr/local/sbin directory.
[Root @ server2 ~] # Cd/usr/local/src/software/sqlsyslogd/
[Root @ server2 sqlsyslogd] # Make
CC-o6-wall-pipe-I/usr/local/include-dconf = \ "/usr/local/etc/sqlsyslogd. conf \ "-l/usr/local/lib/MySQL-lmysqlclient sqlsyslogd. c-o sqlsyslogd
[Root @ server2 sqlsyslogd] # cp sqlsyslogd/usr/local/sbin/
4: run the sqlsyslogd program. If the following command option is displayed, the installation is successful.
[Root @ server2 sqlsyslogd] # sqlsyslogd
Usage: sqlsyslogd [-H hostname] <-u username> [-p] <-T table> [database]

5: Modify the/etc/lD. So. conf file and make it take effect. This file maintains the location of the compiled dynamic link library.
[Root @ server2sqlsyslogd] # Cat/etc/lD. So. conf
Include lD. So. conf. d/*. conf
/Usr/local/lib/MySQL
[Root @ server2 sqlsyslogd] # ldconfig

6. Create a database and a table in the database.
[Root @ server2sqlsyslogd] # MySQL
Welcome to the MySQL monitor. commands end with; or \ G.
Your MySQL connection ID is 158
Server version: 5.1.36-log Source Distribution

Type 'help; 'or' \ H' for help. type' \ C' to clear the current input statement.

Mysql> Create Database syslog;
Query OK, 1 row affected (0.00 Sec)

Mysql> Use Syslog
Database changed
Mysql> Create Table logs (ID int (10) Not null auto_increment, timestampvarchar (16), host varchar (50), prog varchar (50), mesg text, primary Key (ID ));
Query OK, 0 rows affected (0.01 Sec)

Mysql> exit
Bye

7: This file defines the password for connecting to the database
[Root @ server2sqlsyslogd] # Cat/usr/local/etc/sqlsyslogd. conf
123456

8: Add the following configuration in the syslog-ng main configuration file:
[Root @ server2sqlsyslogd] # vi/usr/local/syslog-ng/etc/syslog-ng.conf

Destination sqlsyslogd {
Program ("/usr/local/sbin/sqlsyslogd-u root-T logs syslog-P ");
};

Log {
Source (s_remote );
Destination (sqlsyslogd );
};

9: restart the syslog-ng service.
[Root @ server2sqlsyslogd] # service syslog-ng restart
Stopping kernellogger: [OK]
Starting kernellogger: [OK]

10: client message sending Test
[Root @ server2 sqlsyslogd] # tail-1/var/log/syslog-ng/20100226/192 .168.90.1/messages

Feb 26 14:25:47 192.168.90.1 root [6058]: Just for fun