Open source Distributed search Platform Elk (elasticsearch+logstash+kibana) +redis+syslog-ng realize log real-time search

Source: Internet
Author: User
Tags redis syslog kibana logstash

Turn from:

Logstash + Elasticsearch + kibana+redis+syslog-ng

Elasticsearch is an open source, distributed, restful search engine built on Lucene. Designed for cloud computing, to achieve real-time search, stable, reliable, fast, easy to install and use. Supports the use of JSON for data indexing over HTTP.

Logstash is a platform for application log, event transmission, processing, management, and search. You can use it to unify the collection management of application logs, providing WEB interfaces for querying and counting. In fact, Logstash can be replaced by other, such as common fluented

Kibana is a WEB interface for log analysis provided for Logstash and Elasticsearch. It can be used to efficiently search, visualize, analyze and perform various operations on the log.

Redis is a high-performance memory Key-value database that is not required for installation to prevent data loss.

Http:// Authparam=1408083909_3bf5b46169faab84d36cf74407132bba B0elkelasticsearchlogstashkibana%e5%85%a5%e9%97%a8%e5%ad%a6%e4%b9%a0%e8%b5%84%e6%ba%90%e7%b4%a2%e5%bc%95

IP description
118.X.X.X/16 for Client IP and 61.x.x.x for Elk Intranet and extranet IP installation JDK

Http:// Tar zxvf Jdk-7u67-linux-x64.tar.gz\? authparam\=1408083909_3bf5b46169faab84d36cf74407132b MV Jdk1.7.0_67/usr/local/cd/usr/local/ln-s jdk1.7.0_67 JDK Chown-r Root:root jdk/

Configure Environment variables
Vi/etc/profile Export JAVA_HOME=/USR/LOCAL/JDK export jre_home= $JAVA _home/jre export classpath=.: $JAVA _home/lib/ Dt.jar: $JAVA _home/lib/tools.jar: $JRE _home/lib: $CLASSPATH export path= $JAVA _home/bin: $PATH export redis_home=/usr/ Local/redis Export Es_home=/usr/local/elasticsearch export es_classpath= $ES _home/config

Variable takes effect

Verify version
Java-version java Version "1.7.0_67" Java (tm) SE Runtime Environment (build 1.7.0_67-b01) Java HotSpot (tm) 64-bit Server VM (build 24.65-b04, Mixed mode)

If you have previously installed Java, you can first uninstall
Rpm-qa |grep Java

RPM-E java-1.6.0-openjdk- java-1.6.0-openjdk-devel- installation Redis

Http:// tar zxvf redis-2.6.17.tar.gz MV Redis-2.6.17/usr/local/cd/usr/local ln-s redis-2.6.17 redis cd/usr/local/redis make make install

CD Utils
./ Please select the Redis port for this instance: [6379] Selecting default:6379 Please select the Redis Config file name [/etc/redis/6379.conf] Selected Default-/etc/redis/6379.conf Please select the Redis log file name [/V Ar/log/redis_6379.log] Selected Default-/var/log/redis_6379.log, select the data directory for this instance [/var /LIB/REDIS/6379] Selected Default-/var/lib/redis/6379 Please select the Redis executable path [/usr/local/bin/redis-ser Ver

Edit configuration file
vi/etc/redis/6379.conf daemonize Yes Port 6379 Timeout tcp-keepalive 60

/etc/init.d/redis_6379 start

exists, process is already running or crashed
If you report this mistake, you need to edit the/etc/init.d/redis_6379 to remove \ n on the head.

Join Auto Start
Chkconfig–add redis_6379 Installation Elasticsearch
cluster installation as long as the node in the same network segment, set a consistent, Elasticsearch can be started to detect each other, forming a cluster wget Elasticsearch/elasticsearch/elasticsearch-1.3.2.tar.gz tar zxvf elasticsearch-1.3.2.tar.gz mv elasticsearch-1.3.2/ Usr/local/cd/usr/local/ln-s elasticsearch-1.3.2 elasticsearch elasticsearch/bin/elasticsearch-f [2014-08-20 13:19:05,710][info][node                     ] [Jackpot] Version[1 .3.2], pid[19320], build[dee175d/2014-08-13t14:29:30z] [2014-08-20 13:19:05,727][info][node                     ] [Jackpot] initializing ... [2014-08-20 13:19:05,735] [INFO] [plugins                  [Jackpot] loaded [], sites [] [2014-08-20 13:19:10,722][info][node                     ] [jackpot] initialized [2014-08-20 13:19:10,723][info][node          &NBSP          ] [Jackpot] starting ... [2014-08-20 13:19:10,934] [INFO] [transport                [Jackpot] bound_address {inet[/]}, publish_address {inet[/61.x.x.x:9301]} [2014-08-20 13:19:10,958][INFO ][discovery               ] [Jackpot] ELASTICSEARCH/5HUOX-2ES82S_0ZVI9BUDG [2014 -08-20 13:19:14,011][info][cluster.service         ] [jackpot] New_master [jackpot][ 5HUOX-2ES82S_0ZVI9BUDG][IMPALA][INET[/61.X.X.X:9301]], Reason:zen-disco-join (elected_as_master)

Related Article

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: and provide relevant evidence. A staff member will contact you within 5 working days.

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.