Install Kibana and Logstash under Ubuntu

Original address: http://www.cnblogs.com/saintaxl/p/3946667.html

In short, his specific workflow is to Logstash agent to monitor and filter the log, the filtered log content to Redis (here Redis only processing queues do not store), Logstash Index collects the logs together to the full-text search service Elasticsearch can use Elasticsearch to customize the search by Kibana to combine custom search for page presentation

• Ruby Run Kibana must
• RubyGems Installing Ruby extensions must
• The Bundler function is similar to Yum
• The JDK running the Java program must
• Redis is used to process log queues
• Logstash collecting and filtering logs
• ElasticSearch Full-Text Search service (Logstash integrates one)

Kibana page Show

First to Logstash Index server above, Logstash is divided into index and aget, agent is responsible for monitoring, filtering logs, index is responsible for collecting logs and the log to Elasticsearch to do the search in addition Logstash collection method is divided into Standalone and centralized.

Standalone is all functions on a server, spontaneous self-collection, centralized is collected centrally, a server receives all shipper (personal understanding is Logstash agent) log.

Actually logstash itself does not divide what shipper and collector, just is the configuration file is different, we this time in the centralized way to test

There are two servers here.

192.168.124.128 Logstash INDEX,ELASTICSEARCH,KIBANA,JDK
192.168.124.132 Logstash AGENT,REDIS,JDK

Preparatory work

Installation: OpenSSL

Uninstalling older versions

Apt-get Remove Opensslapt-get autoremove OpenSSL

Download the latest version

wget http://www.openssl.org/source/openssl-1.0.1i.tar.gz

TAR-ZXVF openssl-1.0.1i.tar.gzcd/opt/openssl-1.0.1i./config--prefix=/usr/local/sslmake & make Install

Establishing a soft connection

Ln-s/usr/local/ssl/bin/openssl/usr/bin/opensslln-s/usr/local/ssl/include/openssl/usr/include/openssl

Refresh Dynamic Configuration

Vim/etc/ld.so.conf

Insert a row at the end of a text

/usr/local/ssl/libldconfig-v

Test

OpenSSL version-a

Installing the Pcre Library

wget ftp://ftp.csx.cam.ac.uk/pub/software/programming/pcre/pcre-8.33.tar.gz

TAR-ZXVF pcre-8.33.tar.gzcd pcre-8.33./configure--prefix=/usr/local/pcre-8.33make & make Install

Installing zlib

wget http://zlib.net/zlib-1.2.8.tar.gz

TAR-ZXVF zlib-1.2.8.tar.gzcd zlib-1.2.8./configure--prefix=/usr/local/zlib-1.2.8make & make Install

Installing Nginx

wget http://nginx.org/download/nginx-1.6.1.tar.gz

TAR-ZXVF nginx-1.6.1.tar.gzcd nginx-1.6.1./configure--prefix=/usr/local/nginx--with-http_ssl_module-- With-openssl=/opt/openssl-1.0.1i--with-pcre=/opt/pcre-8.33--with-zlib=/opt/zlib-1.2.8

nginx Command

Start:/usr/local/nginx/sbin/nginx Restart:/usr/local/nginx/sbin/nginx–s Reload stop:/usr/local/nginx/sbin/nginx-s Stop to view the main process: NETSTAT-NTLP check whether it started successfully: Netstat-ano|grep 80

Installing Ruby running Kibana must be

sudo apt-get update  wget http://cache.ruby-lang.org/pub/ruby/2.1/ruby-2.1.2.tar.gz./configure--prefix=/usr/ Local/rubymake && make Install

Environment settings

Vi/etc/environment

Add Ruby's path to the environment variable and save the/etc/environment, as follows:

Path= "/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/ruby/bin"

After modifying the environment variable file, you need to make the modification take effect with the source command as follows:

$ source/etc/environment

To check if the installation was successful, you can enter the following command to test:

$ruby –v

After confirming that the installation is successful and adding a command link through a command, I am not sure what the purpose of creating these links is, and it should be a convention to follow Ruby's "convention is greater than configuration" principle. (Keyboardota)

$ sudo ln-s/usr/local/ruby/bin/ruby/usr/local/bin/ruby$ sudo ln-s/usr/local/ruby/bin/gem/usr/bin/gem

Or:

Apt-get Install Ruby-full

Installing the RubyGems Ruby extension must

wget http://production.cf.rubygems.org/rubygems/rubygems-2.4.1.tgz

TAR-ZXVF RUBYGEMS-2.4.1.TGZCD Rubygems-2.4.1ruby setup.rb

Installing Redis to handle log queues

wget http://download.redis.io/releases/redis-2.8.13.tar.gz

Install the Elasticsearch full-text Search service (Logstash integrates one)

wget https://download.elasticsearch.org/elasticsearch/elasticsearch/elasticsearch-1.3.2.tar.gz

TAR-ZXVF elasticsearch-1.3.2.tar.gzcd elasticsearch-1.3.2 Start:
/usr/local/elasticsearch-1.3.2/bin/elasticsearch-d Access
http://localhost:9200

Install: Logstash Collect, filter logs

wget https://download.elasticsearch.org/logstash/logstash/logstash-1.4.2.tar.gz

TAR-ZXVF logstash-1.4.2.tar.gz

Start

Nohup/usr/local/logstash-1.4.2/bin/logstash-f/usr/local/logstash-1.4.2/agent.conf &

Nohup/usr/local/logstash-1.4.2/bin/logstash-f/usr/local/logstash-1.4.2/indexer.conf &

Vim/usr/local/logstash-1.4.2/agent.conf

Input {  file {    Path = = ["/var/log/*.log", "/var/log/messages", "/var/log/syslog", "/var/log/denyhosts", "/ VAR/LOG/DMESG ","/var/log/faillog ","/var/log/aptitude "]    start_position = = beginning  }  file {    Type = "Nginx-access"    path = "/var/log/nginx/access.log"  }}output {  redis{    host = = " 192.168.124.128 "    data_type =" List "    key =" Logstash "}  }

Vim/usr/local/logstash-1.4.2/indexer.conf

Input {  Redis {    host = "192.168.124.128"    data_type = "List"    key = "Logstash"  }}output {  Elasticsearch {    host = "192.168.124.132" #指定elasticsearch服务位置  }}

Installing Kibana

wget https://download.elasticsearch.org/kibana/kibana/kibana-3.1.0.tar.gz

TAR-ZXVF Kibana-3.1.0.tar.gzvim/usr/local/kibana-3.1.0/config.js

Search for the "Elasticsearch" parameter and modify it to suit your environment:

Elasticsearch: "http://192.168.124.132:9200",

You can also modify the Default_route parameter to open the Logstash dashboard instead of the Kibana Welcome page by default:

Default_route     : '/dashboard/file/logstash.json ',

Download the configuration template

wget https://raw.github.com/elasticsearch/kibana/master/sample/nginx.conf

Modify Nginx Configuration

Vim/usr/local/nginx/conf/nginx.conf

Add server node

    # # Nginx proxy for Elasticsearch + Kibana # In this setup, we is password protecting the saving of DASHBO Ards.    wish to extend the password protection to all paths. # # Even though these paths is being called as the result of an AJAX request, the # Browser would prompt for a usern    Ame/password on the first request # # IF You use this, you'll want to point config.js at Http://FQDN:80/instead of      # http://FQDN:9200 # server {listen *:80;      server_name localhost;      Access_log/usr/local/nginx/logs/kibana.access.log;        Location/{root/usr/local/kibana-3.1.0;      Index index.html index.htm;        } location ~ ^/_aliases$ {Proxy_pass http://127.0.0.1:9200;      Proxy_read_timeout 90;        } location ~ ^/.*/_aliases$ {Proxy_pass http://127.0.0.1:9200;      Proxy_read_timeout 90; } location ~ ^/_nodes$ {Proxy_pass http://127.0.0.1:9,200;      Proxy_read_timeout 90;        } location ~ ^/.*/_search$ {Proxy_pass http://127.0.0.1:9200;      Proxy_read_timeout 90;        } location ~ ^/.*/_mapping {Proxy_pass http://127.0.0.1:9200;      Proxy_read_timeout 90; } # Password protected end points Location ~ ^/kibana-int/dashboard/.*$ {Proxy_pass http://127.0.0.1:920        0;        Proxy_read_timeout 90;          Limit_except GET {Proxy_pass http://127.0.0.1:9200;          Auth_basic "Restricted";        AUTH_BASIC_USER_FILE/USR/LOCAL/NGINX/KIBANA.MYHOST.ORG.HTPASSWD;        }} Location ~ ^/kibana-int/temp.*$ {Proxy_pass http://127.0.0.1:9200;        Proxy_read_timeout 90;          Limit_except GET {Proxy_pass http://127.0.0.1:9200;          Auth_basic "Restricted";        AUTH_BASIC_USER_FILE/USR/LOCAL/NGINX/KIBANA.MYHOST.ORG.HTPASSWD; }      }    }

If a firewall needs to release these ports:

• Port (for the Web interface)
• Port 5544 (to receive remote syslog messages)
• Port 6379 (for the Redis broker)
• Port 9200 (so the Web interface can access Elasticsearch)

Install Kibana and Logstash under Ubuntu