Elk Environment Construction Complete description

Elk Environment Construction Complete description

Elk:elasticserach, Logstash, Kibana three product name of the first letter collection, for the compilation and search of logs. Simple to understand that we can make the service side of the log (Nginx, tomcat, etc.) direct web display view, very convenient.

本机环境说明：    系统：centos7.5    ElasticSerach：6.4.2    Logstash：6.4.2    Kibana：6.4.2    Filebeat：6.4.2

Overview of Deployment

说明：图是直接搜索的网上的，其中nginx、elk这里都采用单机直接部署，所以ip相同，我这里为192.168.21.128。

Process description

 1. 业务请求到达nginx-server机器上的Nginx；  2. Nginx响应请求，并在access.log文件中增加访问记录； 3. FileBeat搜集新增的日志，通过LogStash的5044端口上传日志； 4. LogStash将日志信息通过本机的9200端口传入到ElasticSerach； 5. 搜索日志的用户通过浏览器访问Kibana，服务器端口是5601； 6. Kibana通过9200端口访问ElasticSerach；

Installing the Java Environment

Because the Elasticserach run requires the Java environment support, so the first to configure the Java environment, the specific operation method Baidu bar, online more.

Download Elk installation Package

Elk Official Website: https://www.elastic.co/downloads

Manually download the elk installed tag.gz file, here we put/usr/local/work/under, and unzip, the final effect is as follows:

Create user

Because the Elasticserach runtime does not allow root users, you need to manually create the user and assign permissions, as follows:

1. To create a user group:groupadd elasticsearch
2. To create a user join user group:useradd elasticsearch -g elasticsearch
3. Set the Elasticserach folder for the user Elasticsearch all:chown -R elasticsearch.elasticsearch /usr/local/work/elasticsearch-6.4.2

System Settings section

1. Open the file/etc/security/limits.conf and add the following 4 things:

   * soft nofile 65536* hard nofile 131072* soft nproc 2048* hard nproc 4096

2. Open the file/etc/sysctl.conf and add the following:

   vm.max_map_count=655360

3. Load the SYSCTL configuration and execute the command:sysctl -p

4. Restart your computer and execute the command:reboot

Start Elasticserach

1. Switch to User elasticsearch:su elasticsearch
2. Go to catalogue/usr/local/work/elasticsearch-6.4.2
3. Execute start command: bin/elasticsearch -d , this time will start Elasticsearch in the background (if the start error does not have permission, re-execute the above chown part of the command set permissions)
4. To view the Startup log executable command:tail -f /usr/local/work/elasticsearch-6.4.2/logs/elasticsearch.log
5. Perform the Curl command to check if the service is responding properly: curl 127.0.0.1:9200 receive the following response:

Specific effects:

Configuring Startup Logstash

1. Under Directory/usr/local/work/logstash-6.4.2, create a file default.conf with the following content:

# 监听5044端口作为输入input {    beats {        port => "5044"    }}# 数据过滤filter {    grok {        match => { "message" => "%{COMBINEDAPACHELOG}" }    }    geoip {        source => "clientip"    }}# 输出配置为本机的9200端口，这是ElasticSerach服务的监听端口output {    elasticsearch {        hosts => ["127.0.0.1:9200"]    }}

1. Background Start Logstash Service:nohup bin/logstash -f default.conf –config.reload.automatic &
2. To view the boot log: tail -f logs/logstash-plain.log The successful startup information is as follows:

Kibana

1. Open the Kibana configuration file/usr/local/work/kibana-6.4.2-linux-x86_64/config/kibana.yml, and find this line:

#server.host: "localhost"

Change to the following content:

server.host: "192.168.21.128"

1. Go to Kibana directory:/usr/local/work/kibana-6.4.2-linux-x86_64
2. To execute a startup command:nohup bin/kibana &
3. To view the boot log:tail -f nohup.out
4. In the browser access http://192.168.21.128:5601, see the following page:

注意：    这里访问5601的时候可能访问不通，centos有自己的防火墙及端口限制，具体设置自行百度吧，很多的。    

Configure Filebeat, send Nginx log to Logstash

1. Open the file/usr/local/work/filebeat-6.4.2-linux-x86_64/filebeat.yml, locate and set the location as follows:

1. Continue to modify the Filebeat.yml file, locate the content and set:

1. Start Filebeat:nohup ./filebeat -e -c filebeat.yml -d "publish" &

Verify that Kibana can display Nginx logs

At this point, the environment is completed ~

Elk Environment Construction Complete description