Original address: http://blog.chinaunix.net/uid-11065483-id-3654882.html
Because the company needs to monitor the line record of QQ, originally used the structure of the light +panabit+splunk to do record. Panabit use is quite comfortable, but when the day of the Splunk log records more than 500MB, Splunk free version can no longer use, which makes me very depressed. So I began to look for other solutions, and finally I found the Graylog2. Graylog2 with MongoDB can quickly retrieve log information, which is useful for scenarios where a large number of logs are recorded every day and occasionally require important information to be retrieved.
GRAYLOG2 installed after the following:
The effect is OK, using Quickfilter to quickly retrieve the relevant message. "10,000 years too long, seize", we say dry! Here's how to install GRAYLOG2.
One. Source Package download
Click (here) to collapse or open
- Cd/usr/src
- #mongodb数据库, the primary role here is to store log information
- wget http://fastdl.mongodb.org/linux/mongodb-linux-i686-2.2.1.tgz
- #graylog2日志服务器
- wget http://cloud.github.com/downloads/Graylog2/graylog2-server/graylog2-server-0.9.5.tar.gz
- #yaml是一种编程语言,
- wget http://pyyaml.org/download/libyaml/yaml-0.1.4.tar.gz
- #ruby脚步语言
- wget ftp://ftp.ruby-lang.org//pub/ruby/1.9/ruby-1.9.2-p0.tar.gz
- #graylog2-web Interface
- wget http://cloud.github.com/downloads/Graylog2/graylog2-web-interface/graylog2-web-interface-0.9.5p2.tar.gz
- #eventlog, installing syslog-ng requires installing EventLog
- wget http://www.balabit.com/downloads/files/syslog-ng/sources/3.2.4/source/eventlog_0.2.12.tar.gz
- #syslog-ng Log Server,
- wget http://www.balabit.com/downloads/files/syslog-ng/open-source-edition/3.3.5/source/syslog-ng_3.3.5.tar.gz
- #JDK, Java Environment
- JDK, http://www.oracle.com/technetwork/java/javase/downloads/index.html Open this link to download the appropriate JDK version, Here with Jdk-6u21-linux-i586.bin
You can download it as above, or it can be downloaded to my network, the address is: http://www.kuaipan.cn/file/id_11613978115442246.htm.
Two. Installing dependent packages
Click (here) to collapse or open
- Yum-y Install gcc* openssl-devel glib2-devel
The operating system recommends using the CentOS 5.7 X64 version. When I use the CentOS 6 version, when I go to the gem install bundler, there will be a file not found, Google's solution is not many, but I found that can use the find after the root directory, and then copy the file to the parent directory can be resolved. I have obsessive-compulsive disorder, so I recommend using the CentOS 5.7 X64 version.
Three. Install and configure the MongoDB database
1. Installing MongoDB
Click (here) to collapse or open
- Useradd MongoDB
- mkdir-pv/data/db
- Chown-r Mongodb:mongodb/data
- Tar xvf/usr/src/mongodb-linux-i686-2.2.1.tgz-c/usr/local/
- cd/usr/local/
- LN-SV mongodb-linux-i686-2.2.1 MongoDB
The basic operation of the MONGO can be found in: http://blog.csdn.net/shirdrn/article/details/7105539
2. Provide service startup scripts for MongoDB and start
Click (here) to collapse or open
- #vim/etc/init.d/mongod #内容如下
- ————————————————————————————————
- #!/bin/bash
- # Chkconfig:-90 11
- # DESCRIPTION:MONGODB Server SysV Script
- . /etc/rc.d/init.d/functions
- if [-f/etc/sysconfig/mongod]; Then
- . /etc/sysconfig/mongod
- Fi
- Mongod=/usr/local/mongodb/bin/mongod
- Prog=mongod
- Lockfile=/data/db/mongod.lock
- Retval=0
- options= "--fork--logpath=/data/mongod.log"
- Start () {
- Echo-n $ "Starting $prog:"
- Daemon $mongod $OPTIONS
- Retval=$?
- Echo
- [$RETVAL = 0]
- Return $RETVAL
- }
- Stop () {
- Echo-n $ "Stopping $prog:"
- Killproc-p ${lockfile} $mongod
- Retval=$?
- Echo
- [$RETVAL = 0] && rm-f ${lockfile}
- }
- Case "$" in
- Start
- Start
- ;;
- Stop
- Stop
- ;;
- Restart
- Stop
- Start
- ;;
- Status
- Status-p ${lockfile} $mongod
- Retval=$?
- ;;
- *)
- echo $ "Usage: $prog {start|stop|restart|status|}"
- Exit 1
- Esac
- Exit $RETVAL
3. Modify Permissions + boot from
Click (here) to collapse or open
- chmod A+x/etc/init.d/mongod
- Chkconfig--add Mongod
- Chkconfig Mongod on
- Service Mongod Start
4. Create the required DB instance for GRAYLOG2
Click (here) to collapse or open
- /usr/local/mongodb/bin/mongo
- >use Admin
- >db.adduser (' admin ', ' password ')
- >use graylog2
- >db.adduser (' Graylog ', ' Redhat ')
- >exit
The user name here: "Graylog and Redhat" is behind the GRAYLOG2 connection database that needs to be used. If the subsequent GRAYLOG2 connection MONGO fails, you can use the command: Db.auth ("Graylog", ", Redhat") to add the user to the authentication.
Four. Installing Graylog2-server
1. Installing the JDK Environment
Click (here) to collapse or open
- Cd/usr/src
- chmod a+x Jdk-6u21-linux-i586.bin
- ./jdk-6u21-linux-i586.bin
- MV jdk1.6.0_21//usr/local/
- cd/usr/local/
- LN-SV jdk1.6.0_21 JDK
2. Adding Java Environment variables
Click (here) to collapse or open
- #vim/etc/profile
- ——————————————————
- Java_home=/usr/local/jdk
- Path= $JAVA _home/bin: $PATH
- Export Java_home PATH
- ——————————————————
- #source/etc/profile finally don't forget source
3. Installing Graylog-server
Click (here) to collapse or open
- Cd/usr/src
- TAR-XVF graylog2-server-0.9.5.tar.gz-c/usr/local/
- Useradd Graylog
- Chown-r graylog:graylog/usr/local/graylog2-server-0.9.5/
- Cd/usr/local
- LN-SV graylog2-server-0.9.5 graylog2
- Cp/usr/local/graylog2/graylog2.conf.example/etc/graylog2.conf
4. Configure Garylog2-server
Click (here) to collapse or open
- Vim/etc/graylog2.conf
- ——————————————————————
- Syslog_listen_port = 515 #将默认的514端口修改为其他未使用端口, because the 514 port is the default port of Syslog-ng, it is necessary to use Syslog-ng 514 port to receive logs from other servers, and to handle the
- Syslog_protocol = UDP
- Mongodb_user = Graylog #访问mongodb的用户名
- Mongodb_password = Redhat #访问mongodb用户名的密码
- Mongodb_host = localhost #mogodb的主机
- Mongodb_database = Graylog2 #存储日志的数据库
- Mongodb_port = 27017 #访问mongodb的端口, default is 27017
5. Modify the Graylog-server service startup script configuration file
Click (here) to collapse or open
- Sed-i ' s/java/$JAVA _cmd/'/usr/local/graylog2/bin/graylog2ctl
- Sed-i ' 2 Ajava_cmd=/usr/local/jdk/bin/java '/usr/local/graylog2/bin/graylog2ctl
6. Add Graylog2-server as a system service and start
Click (here) to collapse or open
- VIM/ETC/INIT.D/GRAYLOG2 #内容如下
- —————————————————————————————————
- #!/bin/bash
- #
- # Chkconfig:-83 19
- # description:graylog2-server SysV Script
- #
- Graylog_bin=/usr/local/graylog2/bin
- CD $GRAYLOG _bin &&./graylog2ctl
7. Modify Permissions + self-booting
Click (here) to collapse or open
- chmod a+x/etc/init.d/graylog2
- Chkconfig--add graylog2
- Chkconfig graylog2 on
- Service GRAYLOG2 Start
Five. Installing Graylog2-web-interface
1. Install the Ruby scripting language environment
Click (here) to collapse or open
- Cd/usr/src
- Tar xvf yaml-0.1.4.tar.gz
- CD yaml-0.1.4
- ./configure--prefix=/usr/local/yaml
- Make
- Make install
- Cd/usr/src
- Tar xvf ruby-1.9.2-p0.tar.gz
- CD ruby-1.9.2-p0
- ./configure--prefix=/usr/local/ruby--enable-shared--disable-install-doc--with-opt-dir=/usr/local/yaml
- Make
- Make install
2. Modify the environment variables in path
Click (here) to collapse or open
- Path=......:/usr/local/ruby/bin #在PATH中添加ruby的路径, don't forget to source a bit
3. Installing Graylog2-web-interface
Click (here) to collapse or open
- Cd/usr/src
- Tar xvf graylog2-web-interface-0.9.5p2.tar.gz-c/usr/local/
- cd/usr/local/
- Chown-r graylog:graylog/usr/local/graylog2-web-interface-0.9.5p2/
- LN-SV Graylog2-web-interface-0.9.5p2/graylog2-web-interface
- CD Graylog2-web-interface
- Gem Install bundler #需要连网
- Bundle Install #需要联网
4. Configure Graylog2-web-interface
Click (here) to collapse or open
- Vim/usr/local/graylog2-web-interface/config/mongoid.yml #内容如下
- —————————————————————————————————————————————————————————————————
- Production
- Host:localhost #mongodb的主机
- port:27017 #mongodb的端口号
- Username:graylog #mongodb的用户名
- Password:redhat #mongodb的用户名密码
- DATABASE:GRAYLOG2 #mongodb的数据库名
5. Configure Log Cutting
Click (here) to collapse or open
- Cat >/etc/logrotate.d/graylog2-web-interface <<eof
- /usr/local/graylog2-web-interface/log/*log {
- size=512m
- Rotate 90
- Copytruncate
- Delaycompress
- Compress
- Notifempty
- Missingok}
- Eof
6. Start Graylog2-web-interface
Click (here) to collapse or open
- Cd/usr/local/graylog2-web-interface &&/script/rails server-e Production &
Here I did not learn to do a service script, but to run directly to the machine in the/etc/profile add tmout=0, and then enter the above sentence, it is over. I like the information that keeps scrolling on the screen, it's very technical.
7. Accessing the GRAYLOG2 Web interface
Opening the browser input-->http://server ip:3000<--opens the GRAYLOG2 Web interface. First use basically just add an admin account to be OK.
Six. Install and configure the Syslog-ng service
1. Installing EventLog
Click (here) to collapse or open
- cd/usr/src/
- Tar xvf eventlog_0.2.12.tar.gz
- CD eventlog-0.2.12
- ./configure--prefix=/usr/local/eventlog
- Make
- Make install
2. Installing Syslog-ng
Click (here) to collapse or open
- Cd/usr/src
- Tar xvf syslog-ng_3.3.5.tar.gz
- CD syslog-ng-3.3.5
- Export Pkg_config_path=/usr/local/eventlog/lib/pkgconfig
- ./configure--prefix=/usr/local/syslog-ng
- Make
- Make install
3. Configure Syslog-ng
Click (here) to collapse or open
- Cat >/usr/local/syslog-ng/etc/syslog-ng.conf <<eof
- @version: 3.3
- @include "Scl.conf"
- SOURCE S_local {
- System ();
- Internal ();
- };
- #设置日志的来源为本机udp的514端口
- SOURCE S_network {UDP (IP (0.0.0.0) port (514));};
- #将日志发送到本机的udp515端口进行处理, Port 515 is the port of Graylog2-server,
- Destination d_local {UDP ("127.0.0.1" Port (515));};
- Log {
- Source (s_local);
- Source (s_network);
- Destination (d_local);
- };
- Eof
4. Add As System service
Click (here) to collapse or open
- Vim/etc/init.d/syslog-ng #内容如下
- ____________
- #!/bin/bash
- #
- # Chkconfig:-60 27
- # description:syslog-ng SysV script.
- . /etc/rc.d/init.d/functions
- Syslog_ng=/usr/local/syslog-ng/sbin/syslog-ng
- Prog=syslog-ng
- Pidfile=/usr/local/syslog-ng/var/syslog-ng.pid
- Lockfile=/usr/local/syslog-ng/var/syslog-ng.lock
- Retval=0
- STOP_TIMEOUT=${STOP_TIMEOUT-10}
- Start () {
- Echo-n $ "Starting $prog:"
- Daemon--pidfile= $pidfile $syslog _ng $OPTIONS
- Retval=$?
- Echo
- [$RETVAL = 0] && Touch ${lockfile}
- Return $RETVAL
- }
- Stop () {
- Echo-n $ "Stopping $prog:"
- Killproc-p $pidfile-D $STOP _timeout $syslog _ng
- Retval=$?
- Echo
- [$RETVAL = 0] && rm-f $lockfile $pidfile
- }
- Case "$" in
- Start
- Start
- ;;
- Stop
- Stop
- ;;
- Status
- Status-p $pidfile $syslog _ng
- Retval=$?
- ;;
- Restart
- Stop
- Start
- ;;
- *)
- echo $ "Usage: $prog {start|stop|restart|status}"
- retval=2
- Esac
- Exit $RETVAL
5. Boot from boot
Click (here) to collapse or open
- chmod a+x/etc/init.d/syslog-ng
- Killall syslogd
- Chkconfig--add Syslog-ng
- Chkconfig Syslog-ng on
- Service Syslog-ng Start
At this point, the GRAYLOG2 configuration is complete. Everyone put the syslog to Graylog2 on the spit on it. If there is a failure or error in the installation of the place welcome message.
Finally, to thank http://ant595.blog.51cto.com/blog/5074217/1081094 's blog, I just after the experiment, and then broadcast. It also indicates my operation.
GRAYLOG2+SYSLOG-NG+MONGODB Building Centralized Management log server--reprint