Elk Log System: Filebeat usage and kibana How to set up login authentication

Filebeat is a lightweight, open source shipper for log file data. As the next-generation Logstash forwarder, filebeat tails logs and quickly sends this information to Logstash fo R further parsing and enrichment or to Elasticsearch for centralized storage and analysis.

Filebeat than Logstash seems better, is the next generation of log collectors, ELK (Elastic +logstash + Kibana) later estimated to be renamed EFK.

Filebeat How to use:

1, download the latest filebeat

Address: Https://www.elastic.co/downloads/beats/filebeat then extract to any directory

2, modify the Filebeat.yml file under Filebeat, refer to the following content:

Filebeat:
  prospectors:
    -
      paths:
        -"/var/log/nginx/*.log"
      input_type:log
      Document_type: Nginx-access
 
    -
     paths:
       -"/data/log/order/*.log"
     input_type:log
     Document_type:order-service
 
    -
     paths:
       -"/opt/service/zhifu/logs/*.log"
     input_type:log
     Document_type:zhifu-service
 
output:
  elasticsearch:
    hosts: ["localhost:9200"]
 
logging:
  files:
    rotateeverybytes:10485760

Inside the hosts content, changed to the actual elasticsearch address.

3, set the Elasticsearch filebeat template

1	Curl-xput ' Http://localhost:9200/_template/filebeat?pretty '-d@/etc/filebeat/filebeat. Template.json

Note: The above localhost:9200 changed to the actual Elasticsearch address, followed by a string for the Filebeat root directory Filebeat.template.json The full path, smooth, will return:

1 2 3

{"acknowledged": true}

Indicates that the template has been received.

4. Start

1	. /FILEBEAT-E-C filebeat.yml-d "Publish"

If you can see a bunch of things output, you're sending logs to elastic search. Can browse: Http://192.168.1.111:9200/_search?pretty If a new content is returned, it means OK

After the test is OK, CTRL + C ends, then uses

1	Nohup. /FILEBEAT-E-C filebeat.yml >/dev/null 2>&1 &

Run in the background, and finally into Kibana, create an index, and note that the pattern is: filebeat-*

　　

Second, the Kibana registration authentication problem

Kibana is developed by the Nodejs itself, there is no security restrictions, directly browse the URL can be accessed, if the public network environment is very insecure, can be sent through Nginx request to increase authentication, the method is as follows:

Tips:kibana No restart command, to restart, can only ps-ef|grep node find Nodejs process, kill again.

1, refer to the following content, modify the configuration file:

server {
  listen       ;
  server_name elk.yjmyzz.com;
  Location/{
     auth_basic "secret";
     auth_basic_user_file/data/nginx/db/passwd.db;
     Proxy_pass http://localhost:5601;
     Proxy_set_header Host $host: 5601;
     Proxy_set_header x-real-ip $remote _addr;
     Proxy_set_header x-forwarded-for $proxy _add_x_forwarded_for;
     Proxy_set_header Via "Nginx";
  }
  Access_log off;
}

The above configuration indicates that the elk.yjmyzz.com request is forwarded to port 5601 of the server and is authenticated using the most basic user name and password.

2. Configure login user name, password

1	Htpasswd-c/data/nginx/db/passwd. DB user1

Note passwd.db the path to be consistent with Nginx configuration, the last User1 for the user name, can be arbitrarily changed, after entering the command, the system will prompt for the password, after the passwd.db has encrypted password, interested can be cat to see.

Tip: HTPASSWD is an Apache-brought gadget that attempts to install with Yum install httpd if the command is not found

3, switch off the Kibana port of the extranet access

After using Nginx forwarding, be sure to remember to configure the iptables and other firewalls, prohibit external direct access to port 5601, so it can only be accessed through Nginx.

Reference article:

1, http://elk-docker.readthedocs.org/

2, https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-getting-started.html

3, http://geek.csdn.net/news/detail/54967