Elk Log Real-time analysis system

Logstash:https://download.elastic.co/logstash/logstash/logstash-2.2.2.tar.gz

elasticsearch:https://download.elasticsearch.org/elasticsearch/release/org/elasticsearch/distribution/tar/ Elasticsearch/2.2.0/elasticsearch-2.2.0.tar.gz

Kibana:https://download.elastic.co/kibana/kibana/kibana-4.4.0-linux-x64.tar.gz

Installing the JDK Environment

Yum Install-y java-1.8.0-openjdk

Shutting down the firewall

Service Iptables Stop

Installing Elasticsearch

Elasticsearch to run with a non-root user

Groupadd Hadoop

Useradd-g Hadoop Hadoop

wget https://download.elasticsearch.org/elasticsearch/release/org/elasticsearch/distribution/tar/elasticsearch/ 2.2.0/elasticsearch-2.2.0.tar.gz

Download Plugin

./bin/plugin Install Mobz/elasticsearch-head

./bin/plugin Install Lmenezes/elasticsearch-kopf

Modifying the ELASTICSEARCH.YML configuration

Cluster.name:es_cluster
Node.name:node-1
Path.data:/home/hadoop/elasticsearch/data
Path.logs:/home/hadoop/elasticsearch/logs
network.host:192.168.233.131
network.port:9200

To detect if the installation was successful

Curl Http://192.168.233.131:9200/

Open Http://ip:9200/_plugin/head, Http://ip:9200/_plugin/kopf to view cluster status

Installing Kibana

wget https://download.elastic.co/kibana/kibana/kibana-4.4.0-linux-x64.tar.gz

Modify the KIBANA.YML configuration (mainly modify the IP of the Elasticsearch)

Open ip:5601 to see if the installation was successful

Installing Logstash

wget https://download.elastic.co/logstash/logstash/logstash-2.2.2.tar.gz

Simple Logstash Configuration

Input {stdin{}}
Output {
Elasticsearch {hosts=> ' 192.168.233.131 '}
}

Note: 1. Logstash to have data uploaded to Elasticsearch in order to create an index in Kibana

2, elasticsearch cluster becomes yellow state, can be processed by the following command

Curl-xput "Http://localhost:9200/_settings"-d '
{
"Number_of_replicas": 0
}‘

Elk Log Real-time analysis system