ELK Log System--monitoring Nginx_nginx

Logstash Installation

Download path: Https://www.elastic.co/downloads/logstash (Installation method reference official website installation steps)
To read the Nginx log, configure the Nginx log format
Vim nginx.conf
Modify the Nginx record log format, from the HTTP module

Log_format  main  ' $remote _addr | $time _local | $request | $uri | "
                      $status | $body _bytes_sent | $bytes _sent | $gzip _ratio | $http _referer | "
                      $http _user_agent" | $http _x_forwarded_for | $upstream _addr | $upstream _response_time | $upstream _status | $ Request_time ';

Save after the modification is complete, use the./nginx-s Reload Reload

/ETC/LOGSTASH/CONF.D Create nginx Log configuration file Touch nginx_access.conf sudo vim nginx_access.conf input {file {path =&gt ; ['/usr/local/nginx/logs/adsapi.access.log '] type => ' nginx_access '}} filter {grok {match =&gt ; ["Message", "%{iporhost:clientip} \| %{httpdate:timestamp} \| (?:%{word:verb}%{notspace:request} (?: Http/%{number:http_version})? | -) \| %{uripath:uripath} \| %{number:response} \| (?:%{number:body_bytes_sent}|-) \| (?:%{number:bytes_sent}|-) \| (?:%{notspace:gzip_ratio}|-) \| (?:%{qs:http_referer}|-) \| %{qs:user_agent} \| (?:%{qs:http_x_forwarded_for}|-) \| (%{urihost:upstream_addr}|-) \| (%{base16float:upstream_response_time}) \| %{number:upstream_status} \| (%{base16float:request_time}) "]} geoip {source =>" ClientIP "Target =>" Geoi P "Add_field => [[Geoip][coordinates]", "%{[geoip][longitude]}"] Add_field => ["[Geoip][coordinates]" , "%{[geoip][latitude]} "]} mutate {convert => [[Geoip][coordinates]", "float"} date {match => ["Times Tamp "," Dd/mmm/yyyy:hh:mm:ss Z "]} mutate {Remove_field =>" timestamp "}} output {ELASTICSE Arch {hosts => ["127.0.*.*:9200"] Index => "logstash-nginx-access-%{+yyyy." MM.DD} "User =>" * * * * *//Kibana Setup password => "pwd"} stdout {}}

2. Elasticsearch Installation

Download Address: Https://www.elastic.co/downloads/elasticsearch, installation steps See official website
After the installation is complete, from the etc/elasticsearch/directory
Vim Elasticsearch.yml

Cluster.name:elk
node.name:es2
path.data:/data/elasticsearch (storage directory must be authorized for Elasticsearch account)
Bootstrap.memory_lock:false
bootstrap.system_call_filter:false
network.host: *.*.*.* (server IP)
http.port:9200

Start Services sudo service elasticsearch start
View the startup log, or directly see if the status of the process after startup is successful
Elasticsearch (PID 19206) is running.
Browser input: HTTP://IP address: 9200/, give the response result

3. Kibana Installation

Download Address: Https://www.elastic.co/downloads/kibana
Install x-pack, download address: Https://www.elastic.co/downloads/x-pack
Installed yourself in the/user/share/directory
modifying kibana.yml files from the etc/kibana/directory

sudo vim kibana.yml
server.name: "*.*.*.*"//(server IP address)
elasticsearch.url: "http://*.*.*.*:9200"
Elasticsearch.username: "username"
Elasticsearch.password: "pwd"
added:
tilemap.url: ' http:// WEBRD02.IS.AUTONAVI.COM/APPMAPTILE?LANG=ZH_CN&SIZE=1&SCALE=1&STYLE=7&X={X}&Y={Y}&Z={Z} '

After the configuration is complete, three services start Elasticsearch–kibana–logstash
Service Elasticsearch Start
Service Kibana Start
Initctl Start Logstash
4. Need to configure nginx.conf for extranet access, access address to Kibana

Upstream Elk {
    ip_hash;
    Server 127.0.0.1:5601;
}

server {
    listen;
    server_name domain name;
    Server_tokens off;

    Client_body_timeout 5s;
    Client_header_timeout 5s;

    Location/{
        Proxy_pass http://elk/;
        Index index.html index.htm;
     Proxy_redirect off;
    Proxy_http_version 1.1;
    Proxy_set_header Host $http _host;
    Proxy_set_header Upgrade $http _upgrade;
    Proxy_set_header x-real-ip $remote _addr
    }
}

 5. After the configuration is complete, reload the Nginx, the browser enters the domain name, fills in the installation x-pack username and the password 6. Login successful after! [Write a picture description here] (https://img-blog.csdn.net/20171201115255853?watermark/2/text/aHR0cDovL2Jsb2cuY3Nkbi5uZXQvcWluZ3RpYW4yMDAy/  Font/5a6l5l2t/fontsize/400/fill/i0jbqkfcma==/dissolve/70/gravity/southeast) 7. In the Configure an index-pattern function configuration: logstash-nginx-access*! [Write a picture description here] (https://img-blog.csdn.net/20171201115318889?watermark/2/text/aHR0cDovL2Jsb2cuY3Nkbi5uZXQvcWluZ3RpYW4yMDAy/ Font/5a6l5l2t/fontsize/400/fill/i0jbqkfcma==/dissolve/70/gravity/southeast) 8. After the success of the creation, select the Discover module to find the