CentOS7.2 Deployment ELK5.2.2 (yum installation)

I. Introduction of ELK Open Source real-time log analysis Elk platform can perfectly solve our above problems, elk by Elasticsearch, Logstash and Kiabana three open source tools:Elasticsearch is an open source distributed Search server based on Lucene. It features: distributed, 0 configuration, Auto discovery, Index auto-shard, index copy mechanism, RESTful style interface, multi-data source, automatic search load, etc. It provides a distributed multi-user-capable full-text search engine, based on a restful web interface. Elasticsearch was developed in Java and published as an open source under the Apache license terms, and is the second most popular enterprise search engine. Designed for cloud computing, it can achieve real-time search, stable, reliable, fast, easy to install and use.
In Elasticsearch, the data for all nodes is equal. Logstash is a fully open source tool that you can use to collect, filter, analyze, and store your logs for later use (for example, search). When it comes to search, Logstash comes with a web interface that searches and displays all logs. Kibana is a browser-based page ElasticSearch front-end display tool, also an open source and free tool, it Kibana can provide Logstash and ElasticSearch log analysis of a friendly Web interface, can help you summarize, Analyze and search for important data logs. Deployment Environment: CentOS7.2 Version number 1611,elasticsearch5.2.2,logstash5.2.2,kibana5.2.2. The virtual machine memory is greater than 2G. Turn off firewall and SELinux. 1.java Environment

[Root@elk-node1 ~]# yum install-y java-1.8.0
[root@elk-node1 ~]# java-version
openjdk version "1.8.0_121"
OpenJDK Runtime Environment (build 1.8.0_121-b13)
OpenJDK 64-bit Server VM (build 25.121-b13, Mixed mode)

2.elasticsearch Installation 1. Import Elasticsearch PGP key

[Root@elk-node1 ~]# rpm--import https://artifacts.elastic.co/GPG-KEY-elasticsearch

2. Configure the Yum source and modify the Hosts file

[Root@elk-node1 ~]# Vim/etc/yum.repos.d/elasticsearch.repo
[elasticsearch-5.x]
Name=elasticsearch Repository for5.x Packages
Baseurl=https://artifacts.elastic.co/packages/5.x/yum
gpgcheck=1
gpgkey= Https://artifacts.elastic.co/GPG-KEY-elasticsearch
enabled=1
autorefresh=1
type=rpm-md
[ Root@elk-node1 ~]# echo "192.168.217.131 elk-node1" >>/etc/hosts
[Root@elk-node1 ~]# Echo 192.168.217.132 Elk-node1 ">>/etc/hosts

3. Installing elasticsearch and modifying configuration files

[Root@elk-node1 ~]# yum install elasticsearch-y
[root@elk-node1 ~]# systemctl daemon-reload
[root@elk-node1 ~] # Systemctl Enable Elasticsearch.service   //Add boot
[root@elk-node1 ~]# grep-v ^#/etc/elasticsearch/ ELASTICSEARCH.YML
cluster.name:elk     //Cluster name
node.name:elk-node-1   //node name
path.data:/var/lib/ Elasticsearch   //data Storage path
path.logs:/var/log/elasticsearch   //log storage path
network.host:0.0.0.0    //Listening address
http.port:9200          //Listening port
discovery.zen.ping.unicast.hosts: ["192.168.217.131", " 192.168.217.132 "]   //cluster node discovery list
Discovery.zen.minimum_master_nodes:2   //cluster minimum number
of nodes that can do master Http.cors.enabled:true   
http.cors.allow-origin: "*"

4. Configure the head plugin

[Root@elk-node1 ~]# wget Https://nodejs.org/dist/v6.9.5/node-v6.9.5-linux-x64.tar [root@elk-node1 ~]# TAR-XVF Node-v6.9.5-linux-x64.tar-c/usr/local/[root@elk-node1 ~]# ln-s Node-v6.9.5-linux-x64/bin/node/usr/bin/node [  Root@elk-node1 ~]# ln-s node-v6.9.5-linux-x64/bin/npm/usr/bin/npm [root@elk-node1 ~]# node-v v6.9.5 [Root@elk-node1 ~]# Npm-v 3.10.10 [root@elk-node1 ~]# npm install-g grunt-cli [root@elk-node1 ~]# ln-s Node-v6.9.5-linux-x64/lib/node_modu Les/grunt-cli/bin/grunt/usr/bin/grunt [Root@elk-node1 ~]# cd/var/lib/elasticsearch [Root@elk-node1 elasticsearch]# Yum install-y git [root@elk-node1 elasticsearch]# git clone git://github.com/mobz/elasticsearch-head.git [ Root@elk-node1 elasticsearch]# chown-r elasticsearch:elasticsearch elasticsearch-head/[Root@elk-node1 elasticsearch ]# cd/var/lib/elasticsearch/elasticsearch-head/[root@elk-node1 elasticsearch-head]# npm Install [root@elk-node1 elasticsearch-head]# cd/var/lib/elasticsearch/elasticsearch-head/_site/[roOT@ELK-NODE1 _site]# cp app.js App.js.bak [root@elk-node1 _site]# vim app.js init:function (parent) {
                        This._super (); This.prefs = Services.
                        Preferences.instance (); This.base_uri = This.config.base_uri | | This.prefs.get ("App-base_uri") | |   "Http://192.168.217.131:9200"; Modify Line 4328 [Root@elk-node1 _site]# cd/var/lib/elasticsearch/elasticsearch-head/[Root@elk-node1 elasticsearch-head]#
                        CP gruntfile.js Gruntfile.js.bak [root@elk-node1 elasticsearch-head]# vim gruntfile.js Connect: { Server: {options: {hostname: "0.
                                        0.0.0 ",//Add this line port:9100, base: '. ',
                Keepalive:true}} } [Root@elk-node1 Elasticsearch-head]# systemctl start Elasticsearch.service [root@elk-node1 elasticsearch-head]# grunt Server & [ Root@elk-node1 elasticsearch-head]# netstat-anplut//Check 9200,9300,9100 port

5. Open the browser for verification

Access Http://192.168.217.131:9200/for verification

Access Http://192.168.217.131:9100/for verification

When the Logstash and Kibana are not installed, the Red Line part is blank. 3. Installing Logstash 1.yum Installation

[Root@elk-node1 ~]# yum isntall-y logstash
[root@elk-node1 ~]# cd/etc/logstash/
[Root@elk-node1 logstash]# Grep-v "#" Logstash.yml
path.data:/var/lib/logstash
path.config:/etc/logstash/conf.d
path.logs:/var/ Log/logstash

2.pipeline File according to the default configuration, the pipeline instance file should be placed in the/ETC/LOGSTASH/CONF.D directory by default, when there is no instance file in the directory, you can create a new instance according to the actual situation, in order to handle the native messages information, as an example:

[Root@elk-node1 logstash]# cd/etc/logstash/conf.d/
[root@elk-node1 conf.d]# vim messages.conf
input {
    File {
        path = "/var/log/messages"
    }
}
output {
    Elasticsearch {
        hosts = = [" 192.168.217.131:9200 "," 192.168.217.132:9200 "]
        index =" messages-%{+yyyy. MM.DD} "
    }
    stdout {
# codec = Rubydebug
    }
}
[Root@elk-node1 conf.d]# cd
. [Root@elk-node1 logstash]# chown-r logstash:logstash conf.d/
[root@elk-node1 logstash]# chmod 644/var/log/ Messages

3. Start Validation

1) Start test

[Root@elk-node1 logstash]# cd/usr/share/logstash/
[root@elk-node1 logstash]# bin/logstash-e ' input {stdin {}} out Put {stdout {}} '
warning:could not ' find logstash.yml which are typically located in $LS _home/config Or/etc/logstash . You can specify the path using--path.settings. Continuing using the defaults
Could not a Find log4j2 configuration at path/usr/share/logstash/config/ Log4j2.properties. Using Default Config which logs to console

Logstash can be started, but this kind of authentication method will have alarm, can prompt processing, in "$LS _home" under the "config" directory, and the "/etc/logstash/" under the soft link to the "config" directory, re-execute it, as follows:

[Root@elk-node1 logstash]# mkdir-p/usr/share/logstash/config/
[root@elk-node1 logstash]#  ln-s/etc/ logstash/*/usr/share/logstash/config
[root@elk-node1 logstash]# chown-r logstash:logstash/usr/share/logstash/ config/
[root@elk-node1 logstash]# bin/logstash-e ' input {stdin {}} ' output {stdout {}} '

2) Start Logstash and verify

[Root@elk-node1 ~]# 
systemctl start Logstash
[root@elk-node1 ~]# systemctl enable Logstash.service
Created symlink From/etc/systemd/system/multi-user.target.wants/logstash.service to/etc/systemd/system/ Logstash.service.
[Root@elk-node1 ~]# Netstat-anplut
//View 9600 port

4. Verification

Visit http://192.168.217.131:9100/
4. Installing Kibana 1.yum Mounting Kibana

[Root@elk-node1 ~]# yum install-y kinaba
[root@elk-node1 ~]# cd/etc/kibana
[root@elk-node1 kibana]# grep-v ^# /etc/kibana/kibana.yml 
server.port:5601
server.host: "0.0.0.0"
elasticsearch.url: "/http// 192.168.217.131:9200 "
[root@elk-node1 kibana]# systemctl start Kibana
[Root@elk-node1 kibana]# netstat- Anplut |grep:5601
TCP        0      0 0.0.0.0:5601            0.0.0.0:*               LISTEN      

2. Browser Access Verification

Visit http://192.168.217.131:5601/

Add messages-* in the Red Line section, and then click Discover, if you can't display the graphic, select an hour or longer.
Elk, simply introduce here, what questions are welcome to come forward.