Logstash+es+kibana+redis Construction

Environmental information:

CentOS 6.5

Redis 3.0.4

Logstash

Elasticsearch

Kibana

Service-Side ip:192.168.0.65

Client ip:192.168.0.66

Relationship structure diagram:

Photo citations from: http://www.wklken.me/posts/2015/04/26/elk-for-nginx-log.html

Installing Redis

First download Redis

wget http://download.redis.io/releases/redis-3.0.4.tar.gz

Unzip after download, compile and install

TAR-ZVXF redis-3.0.4.tar.gz

Make

Make install

Then start Redis

Redis-server redis.conf &

Add & sign for background start

Installing the Logstash server side

Installing the Java environment first uses a simpler approach here

Yum Install Jdk–y

If you install manually, it is recommended to install java1.8

Download Logstash

wget https://download.elastic.co/logstash/logstash/logstash-1.5.4.tar.gz

Unzip after download

TAR–ZXVF logstash-1.5.4.tar.gz

Then to the Logstash directory, create a config directory and create a logstash_server.conf file, create a logs directory to hold the log, and create a Logstash.log file

Edit File

Vim logstash_server.conf

Write the content into the logstash_server.conf

Input {
Redis {
Host = "127.0.0.1" #本地的reds地址
Port = 6379 #redis端口
Type = "Redis-input" #输入类型
data_type = "List" #使用redis的list存储数据
Key = "Key_count"
}
}

Output {
StdOut {}
Elasticsearch {
Host = "127.0.0.1" #elasticsearch地址
cluster = "Elasticsearch"
codec = "JSON"
protocol = "HTTP" #elasticsearch使用的协议
}
}

Start Logstash after configuration is complete

Bin/logstash Agent--verbose--config config/logstash_server.conf--log Logs/logstash.log &

Data Acquisition Terminal

The installation steps on the data acquisition side are the same as the server side installation steps

Just the configuration file is different

Create logstash_client.conf on the data acquisition side

The conf file on the data acquisition side is configured as:

input {file {type => "type_count"                  

                 #日志所在的目录

           path => "/data/logs/count/stdout.log", "/data/logs/count/stderr.log"]exclude => ["*.gz", "access.log"]   #排除一些不想加入的日志}   }output {stdout {}redis {host => "192.168.0.65"port => 6379data_type => "list"         #数据类型也要保持一致key => "key_count"          #key 要与服务器端的相同 }   }

Start Logstash after configuration is complete

Bin/logstash Agent--verbose--config config/logstash_server.conf--log Logs/logstash.log &

Installing Elasticsearch

Download Elasticsearch

wget https://download.elastic.co/elasticsearch/elasticsearch/elasticsearch-1.7.2.tar.gz

Decompression: TAR–ZVXF elasticsearch-1.7.2.tar.gz

Elasticsearch does not require a special configuration, it only needs to be started

Cut to Elasticsearch directory to start: Bin/elasticsearch–d

The-d command indicates background startup Elasticsearch

Installing Kibana

Download Kibana

wget https://download.elastic.co/kibana/kibana/kibana-4.1.2-linux-x64.tar.gz

Decompression: TAR–ZXVF kibana-4.1.2-linux-x64.tar.gz

Kibana need to configure the server address where Elasearch resides

Cut to the Kibana directory.

Vim Config/kibana.yml

Find Elasticsearch_url: "http://localhost:9200" This line is modified to elasticsearch_url:http://127.0.0.1:9200
The boot of the Kibana requires a. Kibana hidden file that can be executed after the Kibana is started:

Curl-xput Localhost:9200/.kibana

Test whether the installation was successful

We start the REDIS monitoring input information

Redis-cli–p 6379 Mointor

We then write some test information to the client's log file:

Echo ' This is a test message! ' >>/data/logs/count/stderr.log

You will see the server side receiving the information:

This is where the basic construction is done.

Access to the browser: http://192.168.0.65:5601

Finally, we recommend several Kibana addresses to use:

Http://www.code123.cc/docs/kibana-logstash/v4/index.html

Logstash+es+kibana+redis Construction