1-elk Installation and use tutorial (build log Analysis System)

1, Logstash use

Logstash is a fully open source tool that collects, analyzes, and stores your logs for later use. 1.1 Logstash Installation (JDK 1.8 environment)

Download Logstash 2.3.4, and unzip. 1.2 Logstash Configuration

Configure logstash.conf

Input {
    file {
        type = ' log-file '
        path ' = '/root/test/log/log.log '
    }
 }
output {
    stdout { 
            codec = Rubydebug  
    }
}

Where input is the input source for Logstash, the current configuration is to monitor the Root/test/log/log.log file and set the log type to Log-file.
Where output is a Logstash source, currently configured to output directly to the terminal. 1.3 Logstash Start

./bin/logstash-f logstash.conf

1.4 Logstash UseFirst start the Logstash service to start the printing log program, and the log output to Root/test/log/log.log observation Logstash terminal output, you can see Logstash crawl to the monitoring path of the log, and processing and output.

{
       "message" = "[Info][uid:1491031249][name:gloria harrison][ip:36.59.236.198][date:2017-04-01t15 : 20:49.703+0800] ",
      " @version "=" 1 ",
    " @timestamp "=" 2017-04-01t07:20:50.644z ",
          " path "="/ Root/test/log/log.log ",
          " host "=" 0.0.0.0 ",
          " type "=" Log-file "
}
{
       " message "=" [ Info][uid:14910312497][name:ralph martinez][ip:36.58.103.37][date:2017-04-01t15:20:49.704+0800] ",
      " @version "1",
    "@timestamp" and "2017-04-01t07:20:50.645z",
          "path" = "/root/test/log/log.log",
          " Host "=" 0.0.0.0 ",
          " type "=" Log-file "
}

2, Elasticsearch use

Elasticsearch is a Lucene-based search server. It provides a distributed multi-user-capable full-text search engine, based on a restful web interface. Elasticsearch is a popular enterprise-class search engine developed in Java and published as an open source under the Apache license terms. Designed for cloud computing, it can achieve real-time search, stable, reliable, fast, easy to install and use. 2.1 elasticsearch Installation (JDK 1.8 environment)

Download Elasticsearch 2.3.4, and unzip. 2.2 Elasticsearch Configuration

Configure CONFIG/ELASTICSEARCH.YML

# Host Settings
network.host:172.17.203.210

2.3 Elasticsearch Common plug-in installation Head:is the cluster management tools, data visualization, and the search tool for adding and pruning.

# installation Command
./bin/plugin Install Mobz/elasticsearch-head

Access path: http://localhost:9200/_plugin/head/
- Kopf: is a elasticsearch management tool that also provides APIs for ES cluster operations.

# installation Command
./bin/plugin Install Lmenezes/elasticsearch-kopf

Access path: http://localhost:9200/_plugin/kopf/
- HQ: Monitor and manage Web applications for Elasticsearch instances and clusters.

# installation Command
./bin/plugin Install ROYRUSSO/ELASTICSEARCH-HQ

Access path: http://localhost:9200/_plugin/hq/ 2.4 elasticsearch Boot

./bin/elasticsearch-d-P PID

2.5 Elasticsearch Use 3, Kibana use

Kibana is also an open source and free tool that can help you summarize, analyze, and search for important data logs and provide a friendly web interface. He can provide a WEB interface for Logstash and ElasticSearch log analysis.

Logstash index The log is collected together to the full-text search service Elasticsearch, you can use Elasticsearch for custom search, Kibana to combine custom search for page presentation. 3.1 Kibana installation (JDK 1.8 environment)

Download Kibana 4.5.2, and unzip. 3.2 Kibana Configuration

# Kibana is served by a back end server. This controls the which port to use. The port number of the Web interface

port:5601

# The host to bind the server to. The host host for Web Access

: "127.0.0.1" # The Elasticsearch instance to use for all

your queries. Access path to ES

elasticsearch_url:elasticsearch.url: "http://127.0.0.1:9200"

3.3 Kibana Start

Nohup./bin/kibana &

3.4 Kibana Use 3.4.1 Access Kibana address

127.0.0.1:5601 3.4.2 CREATE INDEX pattern Index contains time-based events: ES index with TIME column use event times to C reate index names
If these 2 are not selected, you can add the index name of ES directly.