Ubuntu14.04 Build Elk Log Analysis System

System operations and developers can use the log to understand the server hardware and software information, check the configuration process errors and the cause of the error occurred. Frequently analyze logs to understand the load of the server, performance security, so as to take timely measures to correct errors. The role of the log is self-evident, but for a large number of logs distributed across multiple machines, viewing is particularly troublesome. Therefore, the use of log analysis system is very necessary for operations personnel.

Open Source Real-time log analyticsELKthe platform can realize log monitoring and analysis,ELKbyElasticSearch,Logstashand theKiabanaThree of open source tools. Official website:https://www.elastic.co/products

• elasticsearch is an open source distributed search engine, it features: distributed, 0 configuration, Auto discovery, Index auto-shard, index copy mechanism, style interface, multi-data source, automatic search load and so on.

• logstash is a fully open source tool that collects, analyzes, and stores your logs for later use (for example, search).

•  Kibanaalso an open source and free tool that heKibanacan be forLogstashand theElasticSearchprovides log analysis friendlyWebinterface to help you summarize, analyze, and search for important data logs.

The workflow is roughly as follows:

650) this.width=650; "Src=" Http://s2.51cto.com/wyfs02/M02/83/03/wKiom1dotUzC0aB5AAFklPK2jjk085.png-wh_500x0-wm_3 -wmp_4-s_4066904769.png "title=" 1.PNG "alt=" Wkiom1dotuzc0ab5aafklpk2jjk085.png-wh_50 "/>

deploy on all services that need to collect logsLogstash, asLogstash Agent(Logstash Shipper) is used to monitor and filter the collection log, sending the filtered content toLogstash Indexer,Logstash Indexercollect logs together to the full-text search serviceElasticSearch, you can useElasticSearchmake a custom search byKibanato combine a custom search for a page presentation.

Elk Platform Construction:

Build based on Ubuntu14.04 64-bit operating system

Download Elk installation Package: https://www.elastic.co/downloads/

650) this.width=650; "src=" Http://s3.51cto.com/wyfs02/M02/83/03/wKioL1dovnSiitJ_AADQh23WfIA459.png "title=" 1.PNG " alt= "Wkiol1dovnsiitj_aadqh23wfia459.png"/>

Download the JDK installation package: http://www.oracle.com/technetwork/java/javase/downloads/jdk8-downloads-2133151.html

650) this.width=650; "src=" Http://s3.51cto.com/wyfs02/M00/83/03/wKioL1dov1KQAVSnAACAdHTl8qk116.png "title=" 1.PNG " alt= "Wkiol1dov1kqavsnaacadhtl8qk116.png"/>

Downloaded packets:

650) this.width=650; "src=" Http://s3.51cto.com/wyfs02/M00/83/03/wKioL1doxV6ijpGCAAAo1FnFW7s990.png "title=" 1.PNG " alt= "Wkiol1doxv6ijpgcaaao1fnfw7s990.png"/>

1. Install the dependent package JDK8:

#sudo Mkdir/usr/lib/jvm#tar xvzf jdk-8u91-linux-x64.tar.gz-c/usr/lib/jvm/

#vim ~/.BASHRC

Append at the bottom of the document

Export Java_home=/usr/lib/jvm/jdk-8u91-linux-x64export Jre_home=${java_home}/jreexport CLASSPATH=.:${JAVA_HOME}/ Lib:${jre_home}/libexport path=${java_home}/bin: $PATH execution: Source ~/.BASHRC

Execute Java-version and Java, with corresponding data and installation done.

2. Install Logstash:

#tar Xvzf logstash-2.3.3.tar.gz

Create the logstash-test.conf configuration file under the logstash-2.3.3 directory, as follows:

#cat Logstash-test.confinput {stdin {}}output {stdout {codec=> Rubydebug}}

Logstash uses input and output to define the relevant configuration of inputs and outputs when the log is collected, in this case, input defines a input,output called "stdin" that defines an output called "stdout". Regardless of what character we enter, Logstash returns the characters we entered in a format where output is defined as "stdout" and uses the codec parameter to specify the Logstash output format.

Start with the following command:

#./bin/logstash agent-f logstash-test.conf

When you start, what you enter on the screen is displayed in the console. If you enter "I want to leave", the display is as follows:

650) this.width=650; "src=" Http://s2.51cto.com/wyfs02/M00/83/04/wKioL1do1N6DHOZKAAAcq2EwIro311.png "title=" 1.PNG " alt= "Wkiol1do1n6dhozkaaacq2ewiro311.png"/>

Indicates that the installation was successful.

3. Install Elasticsearch:

This article is from the "Black Time" blog, so be sure to keep this source http://blacktime.blog.51cto.com/11722918/1791336

Ubuntu14.04 Build Elk Log Analysis System