Elk Deployment Under centos6.5

1. Introduction

Elk is a real-time log analysis platform that provides real-time log analysis for development and operations personnel, facilitating better understanding of system status and code issues.

2, elk in the E (elasticsearch):

(2.1) Install the dependency package first, the official document describes the use of java1.8

Yum-y Install JAVA-1.8.0-OPENJDK

Install Elasticsearch:

Tar zvxf elasticsearch-1.7.0.tar.gz

MV Elasticsearch-1.7.0/usr/local/elasticsearch

Vim/usr/local/elasticsearch/config

CP Elasticsearch.yml Elasticsearch.yml.bak

Vim Elasticsearch.yml (modified)

Cluster.name:elasticsearch

Node.name:syk

Node.master:true

Node.data:true

Index.number_of_shards:5

Index.number_of_replicas:1 (Shard copy)

Path.data:/usr/local/elasticsearch/data

Path.conf:/usr/local/elasticsearch/conf

Path.work:/usr/local/elasticsearch/work

Path.plugins:/usr/local/elasticsearch/plugins

Path.logs:/usr/local/elasticsearch/logs

Bootstrap.mlockall:true (Memory)

Start:/usr/local/elasticsearch/bin/elasticsearch-d

NETSTAT-TLNP View

There will be 9200 and 9300 Java processes

Curl http://192.168.137.50:9200

Show:

{

"Status": 200,

"Name": "Syk",

"Cluster_Name": "Elasticsearch",

"Version": {

"Number": "1.7.0",

"Build_hash": "929b9739cae115e73c346cb5f9a6f24ba735a743",

"Build_timestamp": "2015-07-16t14:31:07z",

"Build_snapshot": false,

"Lucene_version": "4.10.4"

},

"Tagline": "Know, for Search"

}

(2.2) Use the official startup script:

Https://codeload.github.com/elastic/elasticsearch-servicewrapper/zip/master

Use the RZ command to upload to the server

Unzip Elasticsearch-servicewrapper-master.zip

MV elasticsearch-servicewrapper-master/service//usr/local/elasticsearch/bin/

Cd/usr/local/elasticsearch/bin/service

./elasticsearch install (Automatically create a service script under INIT.D)

/etc/init.d/elasticsearch restart

Curl-xget ' Http://192.168.137.50:9200/_count?pretty '-d '

> {

> "Query": {

> "Match_all": {}

>}

>}

> '

will return:

{

"Count": 0,

"_shards": {

"Total": 0,

"Successful": 0,

"Failed": 0

}

}

(2.3) Rest API-based interface (can be deleted and modified)

Install plug-in:/usr/local/elasticsearch/bin/plugin-i elasticsearch/marvel/latest (Automatic installation)

Web Access: Http://192.168.137.50:9200/_plugin/marvel

Installing the cluster Management plug-in

/usr/local/elasticsearch/bin/plugin-i Mobz/elasticsearch-head

Or: Https://github.com/mobz/elasticsearch-head/archive/master.zip downloaded, RZ to the server

Unzip Elasticsearch-head-master.zip

MV Elasticsearch-head-master Plugins/head

Web Access: Http://192.168.137.50:9200/_plugin/head

You can display a fragmented copy of your shard as a Web page.

3, elk in the L (Logstash):

(3.1) Installation Logstash:

i), the official provision of the installation of Yum installation method:

1, RPM--import Https://packages.elastic.co/GPG-KEY-elasticsearch

2, Vim/etc/yum.repos.d/logstash.repo

Add to:

[logstash-2.3]

Name=logstash repository for 2.3.x packages

Baseurl=https://packages.elastic.co/logstash/2.3/centos

Gpgcheck=1

Gpgkey=https://packages.elastic.co/gpg-key-elasticsearch

Enabled=1

3. Yum--enablerepo=logstash-2.3-y Install Logstash

ii), download the TAR package installation:

Tar zvxf logstash-1.5.3.tar.gz

MV Logstash-1.5.3/usr/local/logstash

(3.2) test

/usr/local/logstash/bin/logstash-e ' input {stdin{}} output {Stdout{codec = Rubydebug}} '

Input hehe

Show:

Logstash Startup completed

Hehe

{

"Message" = "hehe",

"@version" = "1",

"@timestamp" = "2016-08-07t17:46:10.836z",

"Host" = "web10.syk.com"

}

This means normal.

(3.3) Write Logstash configuration file

Attention:

Must input{} with output{}

notation: Use = =

Vim/etc/logstash.conf

input{

File {

Path = "/var/log/syk.log"

}

}

output{

File {

Path = "/tmp/%{+yyyy-mm-dd}.syk.gz"

gzip = True

}

}

Start Logstash:/usr/local/logstash/bin/logstash-f/etc/logstash.conf

Cd/var/log

Cat Maillog >> syk.log (Append to Syk.log)

syk.gz compressed files with date names can be seen in/tmp

(3.4) Using Redis storage Logstash:

Yum-y Install Redis (Redis placed on another server)

Vim/etc/redis.conf (modified)

Bind 192.168.137.52

Logstash is also installed on the 192.168.137.52 server

To write a configuration file:

Vim/etc/logstash.conf

input{

File {

Path = "/var/log/syk.log"

}

}

output{

Redis {

data_type = "List"

Key = "System-messages"

Host = "192.168.137.52"

Port = "6379"

db = "1"

}

}

Logstash to start the 52 server:

/usr/local/logstash/bin/logstash-f/etc/logstash.conf

Cd/var/log

Cat Maillog >> syk.log (Append to Syk.log)

Go inside redis. View:

Redis-cli-h 192.168.137.52-p 6379

Select 1

Keys * (You can see system-messages this key)

Llen System-messages (Can see big system-messages the length of this key)

(3.4) Upload the log information collected by Logstash to ES

Write the Logstash configuration file on the 192.168.137.50 server:

Vim/etc/logstash.conf

Input {

Redis {

data_type = "List"

Key = "System-messages"

Host = "192.168.137.52"

Port = "6379"

db = "1"

}

}

Output {

Elasticsearch {

Host = "192.168.137.50"

protocol = "HTTP"

index = "system-messages-%{+yyyy. MM.DD} "

}

}

Start Logstash:

/usr/local/logstash/bin/logstash-f/etc/logstash.conf

When we go to see Redis's Llen system-messages, it turns out to be 0, which means the data has been transferred to ES.

Web Access: http://192.168.137.50:9200/_plugin/head/

Will come up with a system-messages-2016.08.07 copy of the Shard.

4, elk in the K (Kibana):

(4.1) Installation:

Just unzip the MV.

cd/usr/local/kiabna/config/

Vim kibana.yml Modification:

Elastcsearch: "http://192.168.137.50:9200"

Start:

Nohup./bin/kiban & (Default port 5601)

Web Access:

http://192.168.137.50:5601

Related operation needs to cooperate with the picture explanation, here temporarily does not say.

This article is from the "Linux" blog, so be sure to keep this source http://syklinux.blog.51cto.com/9631548/1836732

Elk Deployment Under centos6.5