Syslog-ng Detailed installation configuration

A recent Sendcloud project requires the use of the Syslog-ng collection log.

The principle is multiple production machine deployment of different modules, including Nodejs, Java and so on, each production machine according to a certain rules to generate a log.

Generally our production machine redhat default SYSLOGD logging, you can use SYSLOGD to send logs through the network to the designated log collection machine, unified analysis.

This log collection machine is installed Syslog-ng, the following explains Syslog-ng installation and configuration steps:

[Root@server2 ~]# cd/usr/local/src/tarbag/[Root@server2 tarbag]# wget eventlog/0.2/eventlog_0.2.12.tar.gz [Root@server2 tarbag]# tar-zxvf eventlog_0.2.12.tar.gz-c. /software/[Root@server2 tarbag]# cd ... /software/eventlog-0.2.12/[Root@server2 eventlog-0.2.9]#./configure--prefix=/usr/local/eventlog && Make & amp;& make install [root@server2 eventlog-0.2.9]# ls/usr/local/eventlog/include Lib [root@server2 syslog-ng-3.0. 5]# CD-/usr/local/src/tarbag [root@server2 tarbag]# wget http://www.balabit.com/downloads/files?path=/libol/0.3/ libol-0.3.15.tar.gz [Root@server2 tarbag]# tar-zxvf libol-0.3.15.tar.gz-c. /software/[Root@server2 tarbag]# cd ... /software/libol-0.3.15/[Root@server2 libol-0.3.9]#./configure--prefix=/usr/local/libol && make && make install [Root@server2 libol-0.3.9]# ls/usr/local/libol/bin include Lib [root@server2 libol-0.3.9]# CD-[root@s Erver2 tarbag]# wget Http://www.balabit.com/downloads/files?path=/syslog-ng/open-source-edition/3.3.5/source/syslog-ng_3.3.5.tar.gz [Root@server2 tarbag]# TAR-ZXVF syslog-ng_3.3.5.tar.gz-c. /software/[Root@server2 tarbag]# cd ... /software/syslog-ng-3.3.5/[root@server2 syslog-ng-3.0.5]# export pkg_config_path=/usr/local/eventlog/lib/ Pkgconfig [Root@server2 syslog-ng-3.0.5]#./configure--prefix=/usr/local/syslog-ng--with-libol=/usr/local/libol && make && make install Configure:error:Cannot find EventLog version >= 0.2:is in path?
(If this error occurs, it is basically because the previous Pkg_config_path variable is not specified). [Root@server2 syslog-ng-3.0.5]# ls/usr/local/syslog-ng/bin libexec sbin Share [root@server2 syslog-ng-3.0.5]# Mkdi r/usr/local/syslog-ng/etc [Root@server2 syslog-ng-3.0.5]# Mkdir/usr/local/syslog-ng/var [root@server2 syslog-ng-3.0.5]# CP contrib/syslog-ng.conf.redhat/usr/local/syslog-ng/etc/[Root@server2 syslog-ng-3.0.5]# CP modules.conf scl.conf/usr/local/syslog-ng/etc/[Root@server2 syslog-ng-3.0.5]# CP Contrib/init.d.redhat/etc/init.d/syslog-ng [Root@server2 syslog-ng-3.0.5]# cd/usr/local/syslog-ng/etc/[ Root@server2 etc]# mv Syslog-ng.conf.redhat syslog-ng.conf [root@server2 etc]# cat syslog-ng.conf @version: 3.0 Options {L
Ong_hostnames (off);
Log_msg_size (8192);
Flush_lines (1);
Log_fifo_size (20480);
Time_reopen (10);
Use_dns (yes);
Dns_cache (yes);
Use_fqdn (yes);
Keep_hostname (yes);
Chain_hostnames (no);
Perm (0644);
Stats_freq (43200);
};
SOURCE S_internal {internal ();};
Destination D_syslognglog {file ("/var/log/syslog-ng.log");
 
Log {source (s_internal); destination (d_syslognglog);};
         SOURCE S_local {Unix-dgram ("/dev/log"); 

File ("/proc/kmsg" Program_override ("kernel:")); Filter F_messages {level (info.. Emerg);   };
Defines 7 types of log type filter f_secure {facility (AUTHPRIV);};
Filter F_mail {facility (mail);
Filter F_cron {facility (cron);
Filter F_emerg {level (Emerg);}; Filter F_spooler {level (crit. Emerg) and facility (UUCP, News);
}; Filter F_LOCAL7{Facility (LOCAL7);};   Destination D_messages {file ("/var/log/messages");
Define 7 types of logs at the client location destination d_secure {file ("/var/log/secure");
Destination D_maillog {file ("/var/log/maillog");
Destination D_cron {file ("/var/log/cron");
Destination D_console {usertty ("root");
Destination D_spooler {file ("/var/log/spooler");
Destination D_bootlog {file ("/VAR/LOG/DMESG");
Log {source (s_local); filter (F_emerg); destination (d_console);
Log {source (s_local), filter (f_secure), Destination (d_secure), flags (final);
Log {source (s_local), filter (f_mail), Destination (D_maillog), flags (final);
Log {source (s_local), filter (F_cron), Destination (D_cron), flags (final);
Log {source (s_local); filter (f_spooler); destination (D_spooler);
Log {source (s_local); filter (F_LOCAL7); destination (D_bootlog);
 
Log {source (s_local); filter (f_messages); destination (d_messages);
    # Remote Logging//Defines the listener port source s_remote {TCP (IP (0.0.0.0) port (514));     UDP (IP (0.0.0.0) port (514));
}; Defines the format, location, and permissions that the client log saves on the server destination r_console {file ("/var/log/syslog-ng/$YEAR $month$day/$HOST/console" owner ("
Root ") Group (" Root ") perm (0640) dir_perm (0750) create_dirs (yes); Destination R_secure {file ("/var/log/syslog-ng/$YEAR $month$day/$HOST/secure" owner ("root") group ("Root") perm (0640)
Dir_perm (0750) create_dirs (yes)); Destination R_cron {file ("/var/log/syslog-ng/$YEAR $month$day/$HOST/cron" owner ("root") group ("Root") perm (0640) dir_
Perm (0750) create_dirs (yes)); Destination R_spooler {file ("/var/log/syslog-ng/$YEAR $month$day/$HOST/spooler" owner ("root") group ("Root") perm (
0640) dir_perm (0750) create_dirs (yes)); Destination R_bootlog {file ("/var/log/syslog-ng/$YEAR $month$day/$HOST/bootlog" owner ("root") group ("Root") perm (
0640) dir_perm (0750) create_dirs (yes)); Destination R_messages {file ("/var/log/syslog-ng/$YEAR $month$day/$HOST/messages" owner ("root") group ("Root") perm (
0640) dir_perm (0750) create_dirs (yes)); Log {Source (S_remoTE); Filter (F_emerg); Destination (r_console);
};
Log {source (S_remote), filter (f_secure), Destination (r_secure), flags (final);
Log {source (S_remote), filter (F_cron), Destination (R_cron), flags (final);
Log {source (s_remote); filter (f_spooler); destination (R_spooler);
Log {source (s_remote); filter (F_LOCAL7); destination (R_bootlog);
 
Log {source (s_remote); filter (f_messages); destination (r_messages); [Root@server2 etc]# chmod +x/etc/init.d/syslog-ng [root@server2 etc]# chkconfig--add syslog-ng service Syslog-ng does n  OT Support Chkconfig (if this error occurs, please modify the first four lines of the script as follows) [Root@server2 etc]# head-4/etc/init.d/syslog-ng #!/bin/bash #chkconifg:--add Syslog-ng #chkconfig: 2345 #Description: Syslog-ng The script also needs to modify the following three locations [Root@server2 etc]# grep ' PATH '/etc/init.d/sys Log-ng Path=/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/syslog-ng/bin:/usr/local/syslog-ng/sbin [Root@server2 etc]# grep ' INIT '/etc/init.d/syslog-ng |head-2 init_prog= "/usr/local/syslog-ng/sbin/syslog-ng" # Full path to daemon init_opts= "-f/usr/local/syslog-ng/etc/syslog-ng.conf" * Options passed to daemon [Root@server2 etc]# service syslog-ng start starting Syslog-ng:/usr/local/syslog-ng/sbin/syslog-ng:error while Loadi Ng shared Libraries:libevtlog.so.0:cannot Open Shared object file:no such file or directory starting Kernel Logger: this The error is because the shared library link is not doing well [root@server2 etc]# ln-s/usr/local/eventlog/lib/*/lib/ln-s/usr/local/eventlog/lib/*/lib64/The following question The problem is because the main configuration file is missing: @version: 3.0 This line starting Syslog-ng:configuration file has no version number, assuming Syslog-ng 2.1 format.
 
Please add @version: Maj.min to the beginning of the file; [Root@server2 ~]# service syslog-ng start starting Kernel Logger: [OK] [Root@ser Ver2 etc]# cat/var/log/syslog-ng.log 03:59:07 server2.yang.com syslog-ng[20225]: Syslog-ng starting up;                       version= ' 3.0.5 ' client configuration: [Root@client ~]# tail-1/etc/syslog.conf                               @192.168.90.20 [root@client ~]# logger-i just one test [root@client ~]# Tail-1]/var/log /messages 22:12:02 Client root[2861]: just one Test [root@server2 ~]# cat/var/log/syslog-ng/20100128/192.168.90.1 0/messages 04:24:32 192.168.90.10 root[2861]: Just one Test [root@server2 ~]# CAT/VAR/LOG/SYSLOG-NG/20100128/19 2.168.90.10/secure 04:01:04 192.168.90.10 sshd[2832]: accepted publickey for root from 192.168.90.1 port 48834 ssh 2 04:01:04 192.168.90.10 sshd[2832]: Pam_unix (sshd:session): Session opened in front of user root by (uid=0) configured syslog- Ng, below is a brief overview of how to save the system log in MySQL 1: link MySQL header files and library files to/usr/local [Root@server2 ~]# ln-s/usr/local/mysql/lib/mysql/usr/ Local/lib/mysql [root@server2 ~]# ln-s/usr/local/mysql/include/mysql//usr/local/include [Root@server2 ~]# cd/us
 R/LOCAL/SRC/SOFTWARE/SQLSYSLOGD 2: Download SQLSYSLOGD source package, because it is the entire directory download, so will download the index.html index file

[Root@server2 software]# wget-d-r-np http://www.frasunek.com/sources/security/sqlsyslogd/[Root@server2 software]# CD www.frasunek.com/sources/security/sqlsyslogd/[Root@server2 sqlsyslogd]# rm-rf index.html* [root@server2 sqlsyslogd]# CD contrib/[root@server2 contrib]# rm-rf index.html* [root@server2 contrib]# CDs [ROOT@SERVER2 ~]# MV/USR/L Ocal/src/software/www.frasunek.com/sources/security/sqlsyslogd//usr/local/src/software/3:make, Copy the SQLSYSLOGD binaries to the/usr/local/sbin directory [root@server2 ~]# cd/usr/local/src/software/sqlsyslogd/[root@server2 sqlsyslogd]# make cc-o6-wall-pipe-i/usr/local/include-dconf=\ "/usr/local/etc/sqlsyslogd.conf\"-L/usr/local/lib/ Mysql-lmysqlclient sqlsyslogd.c-o sqlsyslogd [root@server2 sqlsyslogd]# CP SQLSYSLOGD/USR/LOCAL/SBIN/4: Execute under SQLSYSLOGD  program, the following command options indicate successful installation [Root@server2 sqlsyslogd]# sqlsyslogd usage:sqlsyslogd [-H hostname] <-u username> [P] <-t table> [Database] 5: Modify the/etc/ld.so.conf file and make it effective, this file maintains the compiled dynamic link library location [root@server2 sqlsyslogd]# cat/etc/ld.so.conf include ld.so.conf.d/*.conf/usr/local/lib/mysql [Root@server2 sqlsyslogd]# ldconfig 6: Database to create appropriate libraries and tables [Root@server2 sqlsyslogd]# MySQL Welcome to the MySQL monitor. Commands End With;
or \g. Your MySQL Connection ID is 158 Server version:5.1.36-log Source distribution Type ' help '/' \h ' for help.
 
Type ' \c ' to clear the current input statement.
mysql> CREATE DATABASE syslog; Query OK, 1 row Affected (0.00 sec) mysql> use syslog Database changed mysql> CREATE TABLE logs (Id int (a) not NU
LL auto_increment,timestamp varchar (), Host varchar (), Prog varchar (m), MESG text,primary KEY (ID); Query OK, 0 rows affected (0.01 sec) mysql> Exit Bye 7: This file defines the password for the connection database [Root@server2 sqlsyslogd]# cat/usr/local/etc /sqlsyslogd.conf 123456 8: Add the following configuration in the Syslog-ng main configuration file [Root@server2 sqlsyslogd]# vi/usr/local/syslog-ng/etc/
 
syslog-ng.conf Destination sqlsyslogd{Program ("/usr/local/sbin/sqlsyslogd-u root-t logs syslog-p"); Log {sOurce (S_remote);
Destination (SQLSYSLOGD);
 
};                                    9: Restart Syslog-ng services [root@server2 sqlsyslogd]# service syslog-ng restart stopping Kernel:
[OK] Starting Kernel Logger: [OK] 10: Client sends message test [Root@server2 sqlsyslogd]# tail-1/var/lo G/syslog-ng/20100226/192.168.90.1/messages Feb 14:25:47 192.168.90.1 root[6058]: Just for fun

The configuration of the syslog-ng.conf file in the installation step above

is not as good as the image, and a syslog-ng.conf simple configuration is attached below for reference:

############################################################################# # Default syslog-ng.conf file which
Collects all local logs to a # single file called/var/log/messages.
        # @version: 3.3 @include "scl.conf" source s_local {System ();
Internal ();

};
         SOURCE S_network {TCP (IP (0.0.0.0) port (514));
UDP (IP (0.0.0.0) port (514));

}; Filter F_outbound {level (info..
         Emerg);
Host ("Ip|ip") and program ("^node-syslog-outbound");

};

Destination d_local {file ("/var/log/messages");

Destination D_network {file ("/opt/apps/log/$YEAR $month$day$hour$min.log" Create_dirs (yes));
        Log {#source (s_local);
        #destination (d_local);
        Source (s_network);
        Filter (F_outbound);
Destination (d_network);


}; Filter F_inbound {level (info..
         Emerg);
Host ("Ip|ip") and program ("^node-syslog-inbound");

}; Destination D_inbound {file ("/opt/apps/log_inbound/$YEAR $month$daY$hour$min.log "Create_dirs (yes)); };

Log {
        source (s_network);
        Filter (f_softbound);
        Destination (d_softbound);
};

The above configuration of the log collection machine, but other production machines how to send the log over.

Because other production machine defaults to use SYSLOGD write syslog, and syslogd from send log to the network, only need to add configuration.

Add a row to the/etc/syslog.conf:

*.*             @ip

SYSLOGD will send the Sysylog on the server to this IP, the default connection port is 514, which can be seen in the Syslog-ng configuration.

OK, a log collection is completed.