Lamp+logzilla2.9.9+syslog-ng implementation of centralized log management (second Edition) _ Log

Source: Internet
Author: User
Tags mysql host syslog system log
Liu Tians, 2010/07/24 14:28, Linux, Comments (67), reading (40557), Via site Original Big | In | A preface
Why to introduce the second edition. Since the first edition is based on Logzilla3.0, the author has licensed restrictions after Logzilla3.0, which can be confirmed from the content of the author's reply to the message [Figure 1]. Therefore, it is necessary to update license.txt regularly to achieve the purpose of prolonging the use of time, personal feeling more troublesome, but also the number of host and log restrictions, which is the reason for the second edition, of course, you can also access to the following channels free, unrestricted licensed, see How to get a Free, unlimited, license of Logzilla. But this is not the topic of this article:), the second edition of my use of logzilla2.9.9 version to build a free, unlimited log centralized management platform, functional and 3.0 difference is not small, well, nonsense less say, let's start.
[Figure 1]


Platform screenshot



Second, platform initialization
Deploy Lamp Platform (abbreviated)
Reference
#yum Install libdbi* libnet
#cpan Date::calc Text::levenshteinxs STRING::CRC32
#cpan-I. DIGEST::SHA1
#cpan-I. Net::mysql

Third, download the related package
Reference
#cd/home/install
#mkdir LOGZILLA;CD Logzilla
#wget http://www.balabit.com/downloads/files/eventlog/0.2/eventlog_0.2.9.tar.gz
#wget http://www.balabit.com/downloads/files/syslog-ng/open-source-edition/3.0.3/setups/rhel-5-i386/ syslog-ng-3.0.3-1.rhel5.i386.rpm

Iv. Start Installation
Reference
# CP Eventlog_0.2.9.tar.gz/usr/src/redhat/sources/
# tar ZXVF eventlog_0.2.9.tar.gz
# CD eventlog-0.2.9/
# Rpmbuild--ba eventlog.spec.bb
# cd/usr/src/redhat/rpms/x86_64
# RPM-UVH libevtlog*

#cd/home/install/logzilla
#rpm-UVH syslog-ng-3.0.3-1.rhel5.i386.rpm

V. Installation of Logzilla
Reference
#cd/www/webroot/
#wget http://php-syslog-ng.googlecode.com/files/logzilla_v2.9.9o.tgz
#tar XZVF logzilla_v2.9.9o.tgz
#mkdir-P/var/log/logzilla

VI. Configure MySQL
Reference
# Mysql-u-P
mysql> SELECT @ @event_scheduler;
+-------------------+
| @ @event_scheduler |
+-------------------+
| Off |
+-------------------+
1 row in Set (0.00 sec)


Below the Mysql5.1 version will be prompted as follows, if you do not intend to upgrade to logzilla3.0, there is no relationship, you can skip this step.
mysql> SELECT @ @event_scheduler;
ERROR 1193 (HY000): Unknown system variable ' Event_scheduler '

Activate Event_scheduler
Reference
mysql> SET GLOBAL event_scheduler = 1;
Query OK, 0 rows Affected (0.00 sec)

mysql> SELECT @ @event_scheduler;
+-------------------+
| @ @event_scheduler |
+-------------------+
| On |
+-------------------+
1 row in Set (0.00 sec)

Mysql> quit;

Vii. Modify SYSLOG-NG Configuration
#vi/opt/syslog-ng/etc/syslog-ng.conf
View plain print? @version: 3.0   source s_local {   Internal ();   Unix-stream ("/dev/ Log ");   file ("/proc/kmsg " program_override (" kernel:  "));  };   Source  s_local {           udp (IP (0.0.0.0)  port (514) );  };      # destinations   destination d_messages {  file ("/var/log/messages"); };         ################################## #########################################################   # clay ' S LogZilla config  below   ####################################################################################### ####   # last updated on 2010-06-15   ################################## #########################################################   OPTIONS&NBSP;{&NBSP;&NBsp       long_hostnames (off);         #  doesn ' T actually help on solaris, log (3)  truncates at 1024  chars         log_msg_size (8192);          # buffer just a little for performance          # sync (1);  <- deprecated - use flush_lines ()  instead          flush_lines (1);          # memory is cheap, buffer messages unable to write  (like  to loghost)          log_fifo_size (16384);          # hosts we don ' t want syslog from          #bad_hostname ("^" (ctld.| Cmd|tmd|last) $ ");        # the time to wait before  a dead connection is reestablished  (seconds)          time_reopen (a);         #Use  dns so that our  good names are used, not hostnames        use _dns (yes);        dns_cache (yes);        # use the whole dns name        use_fqdn (yes);         keep_hostname (yes);        chain_hostnames (NO) ;         #Read  permission for everyone         perm (0644);        # the default action of syslog-ng 1.6.0 is to log a stats  line        # to the file every 10 minutes.   that ' s pretty ugly after a while.          # Change it to every 12 hours so you get a  nice daily update of         # # how many  messages syslog-ng missed  (0) .         #  Stats (43200);  };      destination d_logzilla {       program ("/www/webroot/php-syslog-ng/scripts/db_insert.pl"       template ("$HOST \ t $FACILITY \t$priority\t$level\t$tag\t$year-$MONTH-$DAY \t$hour: $MIN: $SEC \t$program\t$msg\n ")        teMplate_escape (yes)       );  };      # tell  syslog-ng to log to our new destination    log {      source (s_local);         destination (D_logzilla);   };  
Eight, modify Apache configuration
#vi httpd.conf
View plain print? <VirtualHost *:80>       ServerAdmin liutiansi@gmail.com        DocumentRoot /www/webroot/php-syslog-ng/html/        ServerName syslog.com.cn       errorlog logs/syslog.com.cn-error_ log       CustomLog logs/syslog.com.cn-access_log common   #  LogZilla   alias /logs  "/www/webroot/php-syslog-ng/html/"    <directory   "/www/webroot/php-syslog-ng/html/" >       Options Indexes  multiviews followsymlinks       AllowOverride All            Order allow,deny            Allow from all   </Directory>   &LT;/VIRTUALHOST&GT;&NBsp; 
Restart the Apache service:/etc/init.d/apache2 restart

Ix. Modification of PHP.ini
Vi/usr/local/php/lib/php.ini
Memory_limit = 128M Max_execution_time = 300/etc/init.d/apache2 restart
X. Configuring log-delimited
Cp/www/webroot/php-syslog-ng/scripts/contrib/system_configs/logrotate.d/etc/logrotate.d/logzilla

Xi. adding jobs
Reference
@daily/usr/local/php/bin/php/www/webroot/php-syslog-ng/scripts/logrotate.php >>/var/log/php-syslog-ng/ Logrotate.log
@daily/usr/bin/find/www/webroot/php-syslog-ng/html/jpcache/-atime 1-exec rm-f ' {} ';
0,5,10,15,20,25,30,35,40,45,50,55 * * * */usr/local/php/bin/php/www/webroot/php-syslog-ng/scripts/reloadcache.php >>/var/log/php-syslog-ng/reloadcache.log

12. Start Platform Installation
#/etc/init.d/syslog-ng Restart
http://192.168.0.100/logs/for installation
The steps are as follows:


















13. Replace Script path
Reference
Cd/www/wewroot/php-syslog-ng/scripts
./fixpaths.sh

or simply use SED to modify
Reference
#sed-I-e "{s@/path_to_logzilla@/www/webroot/php-syslog-ng@}" *.*

#/etc/init.d/syslog-ng restart

14. Client Configuration
In the last add the following, where syslog.admin.com.cn is the host domain name, can also be directly replaced by IP.
#vi/etc/syslog.conf
*.emerg;*.err;*.warning @syslog. admin.com.cn
#/etc/init.d/syslog restart

Test: Logger-p Local4.err "This was a Local.err test message."

Ii. Follow-up questions
When you click [Graph], you will be prompted "Jpgraph Error Font file"/usr/share/fonts/truetype/msttcorefonts/verdana.ttf "is not readable or does Not exist. "
Workaround:
Mkdir-p/usr/share/fonts/truetype/msttcorefonts/
Upload the font files under Windows xp/2003/vista/Verdana.ttf to/usr/share/fonts/truetype/msttcorefonts/.

Problem two, logs table no data, run/www/webroot/php-syslog-ng/scripts/contrib/dbgen/dbgen.pl
Hint: cannot determine peer address at/usr/lib/perl5/site_perl/5.8.5/net/mysql.pm line 277

Workaround:
1, is a logzilla2.9.9 installation of a bug, the new MySQL syslogadmin, sysloguser user rights are not successfully given, manually added on the OK, thanks to "Hangzhou" Figo provided the fault case.
2, such as the 1th step is unsuccessful, try to modify the/www/webroot/php-syslog-ng/html/config/config.php file MySQL host address localhost 127.0.0.1.

Question three, the search cache chart displays only two days of data, with no data in the last three days.
Workaround:
Because the Search_cache table uses the memory storage engine, has the size limit, modifies the/etc/my.cnf, adds in [MYSQLD]:
tmp_table_size=1g
Max_heap_table_size = 1G
You can restart MySQL again.
Effect Chart:


Problem four, can not display the log with "<", ">" issues, such as Cisco System log.
thank [Changzhou] Old high provide the fault case.
Workaround:
Find in tailresult.php and regularresult.php
View plain print?           if (cisco_tag_parse) {$row [' msg '] = preg_replace ('/\s:/', ': ', $row [' msg ']);       $row [' msg '] = preg_replace ('/.* (%.*?:.*)/', ' $ ', $row [' msg ']); }
Add in the inside
View plain print?    $row [' msg '] = preg_replace ('/</', ', ', $row [' msg ']); $row [' msg '] = preg_replace ('/>/', ', ', $row [' msg ']);
Can.

Problem five, the client log is not available for storage.
Thank you [Beijing] for always flying the failure case.
System environment:
CentOS Release 5.5 (Final) i386 2.6.18-194.el5
LAMP (System rpm default installation)
mysql:5.0.77
php:php 5.1.6
apache:2.2.3

Reason:
Perl for MySQL driver exception, directly in the Bin/mysql+insert SQL statement storage.

Workaround:
#vi/opt/syslog-ng/etc/syslog-ng.conf
View plain print? @version: 3.0   #Default  configuration file for syslog-ng.      # for a description of syslog-ng configuration file  directives, please read   # the syslog-ng administrator?ˉs guide  at:   #   # http://www.balabit.com/dl/html/syslog-ng-admin-guide_en.html/bk01-toc.html    #   options {   # number of syslog lines stored  in memory before being written to files   flush_lines  (1);     log_fifo_size  (2048);   create_dirs  (yes);   perm  (0640);    dir_perm  (0750);   keep_hostname  (yes);              time_reopen  (a);             log_fifo_size  (1024);              long_hostnames  (off);              use_dns  (NO);             use_fqdn  (NO);             owner (Root);              group (Root);             perm (0640);  };    source s_network_1 {   UDP (IP ( 0.0.0.0)  port (514));  };   destination d_network_1 {   file ("/var/ log/syslog-ng/network/$YEAR. $MONTH. $DAY/$HOST/$FACILITY. Log ");  };   # define the  destination  "d_network_1b"  log directory   destination d_network_1b {  &nbSp file  ("/var/log/syslog-ng/network/all/network.log");  };   log {   Source (s_network_1);   destination (d_network_1);  };   Source (s_network_1);   destination (d_network_1b);  };   logzilla {   Program ("/var/www/html/php-syslog-ng/scripts/db_insert.pl"     Template ("$HOST \t$facility\t$priority\t$level\t$tag\t$year-$MONTH-$DAY \t$hour: $MIN: $SEC \t$program\t$msg\n"));   };#   log {   Source (s_network_1);   destination (D_logzilla);   };   source localhost_all {   Internal ();   Unix-stream ("Dev /log ");   file ("/proc/kmsg " program_override (" kernel ");  };    localhostlog {   file  ("/var/log/syslog-ng/$HOST/$YEAR-$MONTH/$DAY"  create_dirs ( YES)); &NBSP;&NBSp };   log {   Source (localhost_all);   destination (localhostlog);  };    destination local_logzilla {   Program ("/var/www/html/php-syslog-ng/scripts/db_ insert.pl "    template (" $HOST \t$facility\t$priority\t$level\t$tag\t$year-$MONTH-$DAY \t$hour: $MIN : $SEC \t$program\t$msg\n "));  };   destination d_mysql {        program ("/usr/bin/mysql -usyslogadmin -psyslogadmin syslog"         template ("insert into logs " (host, facility, priority, level,  TAG,&NBSP;DATETIME,&NBSP;PROGRAM,&NBSP;MSG,&NBSP;SEQ)       VALUES  (  ' $HOST ',   ' $FACILITY ',  ' $PRIORITY ',  ' $LEVEL ',  ' $TAG ',  ' $YEAR-$MONTH-$DAY   $HOUR: $MIN: $SEC ' ,  ' $PROGRAM ',  ' $MSG ',  ' $SEQ '  ); \ n ')        template-escape (Yes);       };   log {   Source (localhost_all);    Destination (Local_logzilla);  };  
* Note: The relevant parameters in the configuration should be modified according to the actual circumstances of the individual.


The world is looking for logzilla_v2.9.9o.tgz bag, pack the bag on the line to put up (fix the original small bug)
Download file (downloaded 1021 times) Click here to download the file: logzilla_v2.9.9o.tgz

References: Http://nms.gdd.net/index.php/LogZilla_Installation_Guide

If you have any questions or interested in the topic can be Weibo with me through the exchange: Http://t.qq.com/yorkoliu Tags:logzilla, syslog, Syslog-ng, log

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: info-contact@alibabacloud.com and provide relevant evidence. A staff member will contact you within 5 working days.

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.