Centos7 install ELK and centos7 install elk

Centos7 install ELK and centos7 install elk
1. Overview

• ELK Introduction
  ELK is short for Elasticsearch + Logstash + Kibana:

  Elasticsearch is a Lucene-based search server. It provides a distributed full-text search engine with multi-user capabilities, developed based on java
  Logstash is a tool for receiving, processing, and forwarding logs.
  Kibana is a browser-based front-end Elasticsearch display tool. Kibana is all written in HTML and Javascript.

• Operating system version

cat /etc/redhat-release CentOS Linux release 7.0.1406 (Core) 

2. System Configuration

• Disable selinux

 sed -i "s/SELINUX=enforcing/SELINUX=disabled/" /etc/selinux/config setenforce 0

• Set firewall
  Install firewall (if not, install it first)

yum install firewalld firewall-configsystemctl start firewalld.servicesystemctl enable firewalld.servicesystemctl status firewalld

Port to be opened

Service	Port to be opened
Elasticsearch	Tcp/9200 and 9300
Kibana	Tcp/5601
Logstash	Tcp/5000

firewall-cmd --permanent --add-port={9200/tcp,9300/tcp}firewall-cmd --permanent --add-port=5601/tcpfirewall-cmd --permanent --add-port=5000/tcpfirewall-cmd --reloadfirewall-cmd --statefirewall-cmd --list-all

• Set FQND

#cat /etc/hostname        elk#cat /etc/hosts127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4::1         localhost localhost.localdomain localhost6 localhost6.localdomain610.0.0.102 elk.zll.com  elk#hostname -F /etc/hostname#hostname -f               elk.zll.com

3. Install Elasticsearch

Yum install java-1.7.0-openjdk (install java) wget https://download.elastic.co/elasticsearch/elasticsearch/elasticsearch-1.7.1.noarch.rpmyum-y localinstall elasticsearch-1.7.1.noarch.rpm

Start the service

systemctl daemon-reloadsystemctl enable elasticsearch.servicesystemctl start elasticsearchsystemctl status elasticsearch

View the Elasticsearch configuration file

rpm -qc elasticsearch/etc/elasticsearch/elasticsearch.yml/etc/elasticsearch/logging.yml/etc/init.d/elasticsearch/etc/sysconfig/elasticsearch/usr/lib/sysctl.d/elasticsearch.conf/usr/lib/systemd/system/elasticsearch.service/usr/lib/tmpfiles.d/elasticsearch.conf

View Elasticsearch log files

View the Elasticsearch port (allow in firewall)

4. Install kibana

Download Software Package

wget https://download.elastic.co/kibana/kibana/kibana-4.1.1-linux-x64.tar.gztar zxf kibana-4.1.1-linux-x64.tar.gz -C /usr/local/cd /usr/local/mv kibana-4.1.1-linux-x64 kibana

Create a kibana. service Startup File.

cat > /etc/systemd/system/kibana.service <<EOF[Service]ExecStart=/usr/local/kibana/bin/kibana[Install]WantedBy=multi-user.targetEOF

Start the kibana Service

systemctl enable kibanasystemctl start kibanasystemctl status kibana

View the kibana Port

Web Input http: // ip_address: 5601

5. Install Logstash

Install software package

wget https://download.elastic.co/logstash/logstash/packages/centos/logstash-1.5.3-1.noarch.rpmyum localinstall logstash-1.5.3-1.noarch.rpm 

Set ssl
Use FQDN to create an SSL Certificate (for example, elk.zll.com)

cd /etc/pki/tlsopenssl req -subj '/CN=elk.zll.com/' -x509 -days 3650 -batch -nodes -newkey rsa:2048 -keyout private/logstash-forwarder.key -out certs/logstash-forwarder.crt

Create a file 01-logstash-initial.conf

cat > /etc/logstash/conf.d/01-logstash-initial.conf << EOFinput {  lumberjack {    port => 5000    type => "logs"    ssl_certificate => "/etc/pki/tls/certs/logstash-forwarder.crt"    ssl_key => "/etc/pki/tls/private/logstash-forwarder.key"  }}filter {  if [type] == "syslog" {    grok {      match => { "message" => "%{SYSLOGTIMESTAMP:syslog_timestamp} %{SYSLOGHOST:syslog_hostname} %{DATA:syslog_program}(?:\[%{POSINT:syslog_pid}\])?: %{GREEDYDATA:syslog_message}" }      add_field => [ "received_at", "%{@timestamp}" ]      add_field => [ "received_from", "%{host}" ]    }    syslog_pri { }    date {      match => [ "syslog_timestamp", "MMM  d HH:mm:ss", "MMM dd HH:mm:ss" ]    }  }}output {  elasticsearch { host => localhost }  stdout { codec => rubydebug }}EOF

Start the logstash Service

Systemctl restart logstashsystemctl status logstashchkconfig logstash on (special settings for startup)

Check logstash logs for errors
Tail/var/log/logstash. log

View the logstash port (enabled in firewall)

6. Install Logstash Forwarder on the client

• Install software package

wget https://download.elastic.co/logstash-forwarder/binaries/logstash-forwarder-0.4.0-1.x86_64.rpmyum localinstall logstash-forwarder-0.4.0-1.x86_64.rpm

• Modify the profile/etc/logstash-forwarder.conf
  Modify elk-server in the configuration file

cp /etc/logstash-forwarder.conf /etc/logstash-forwarder.conf.oldcat > /etc/logstash-forwarder.conf << EOF{  "network": {    "servers": [ "elk.zll.com:5000" ],    "ssl ca": "/etc/pki/tls/certs/logstash-forwarder.crt",    "timeout": 15  },  "files": [    {      "paths": [        "/var/log/messages",        "/var/log/secure"      ],      "fields": { "type": "syslog" }    }  ]}EOF

• Start the service and set enable startup

systemctl restart logstash-forwarderchkconfig logstash-forwarder onsystemctl status logstash-forwarder

Log on to the elk-server web page and configure: http: // ip_address: 5601

Reference: Chen shake log

Copyright Disclaimer: This article is an original article by the blogger and cannot be reproduced without the permission of the blogger.