First, Introduction
1. Core composition
ELK Consists of three parts: Elasticsearch,Logstash and Kibana ;
Elasticsearch is an open source distributed search engine, it features: distributed, 0 configuration, automatic discovery, Index auto-shard, index copy mechanism, RESTful style interface, multi-data source, automatic search load, etc.
Logstash is a fully open source tool that collects, analyzes, and stores your logs for later use
Kibana is an open source and free tool that provides log analytics friendly Web interface for Logstash and ElasticSearch to help you summarize, analyze, and search for important data logs.
2. Four components
Logstash: Logstash server side used to collect logs;
Elasticsearch: Store all kinds of logs;
Kibana: The Web interface is used as a search and visualization log;
Logstash forwarder: Logstash client is used to send logs to Logstash server via Lumberjack Network protocol;
3. Elk Work Flow
Deploy Logstash on all services that need to collect logs, as Logstash agent (Logstash shipper) to monitor and filter the collection logs, send filtered content to Redis, and Logstash Indexer collect the logs together to the full-text search service Elasticsearch, you can use Elasticsearch to customize the search by Kibana to combine custom search for page presentation.
650) this.width=650; "src=" http://s3.51cto.com/wyfs02/M01/74/25/wKioL1YWGBnxpAwyAAGFWRpT6X8070.jpg "title=" work Flying.png "alt=" Wkiol1ywgbnxpawyaagfwrpt6x8070.jpg "/>
4, Elk's Help manual
Elk Official Website: https://www.elastic.co/
Elk Official Documents: https://www.elastic.co/guide/index.html
Elk Chinese Manual: http://kibana.logstash.es/content/elasticsearch/monitor/logging.html
Comments
Elk has two mounting options
(1) Integrated environment: Logstash has an integration package that includes its full set of three components, which is the installation of an integration package.
(2) Independent environment: three components are individually installed, run, and perform their respective duties. (more commonly used)
Two. Installation configuration
1. Environment configuration
server1:172.25.29.1 Master
server2:172.25.29.2 data
server3:172.25.29.3 data
2. Installation
650) this.width=650; "src=" Https://s1.51cto.com/oss/201711/07/eadd5c5b28ba462c23df0082e8796b37.png "style=" float: none; "Title=" screenshot from 2017-11-07 09-44-53.png "alt=" Eadd5c5b28ba462c23df0082e8796b37.png "/>
Installing the Java Environment
650) this.width=650; "src=" Https://s1.51cto.com/oss/201711/07/b99c82804f65301409d1e9083dc4a4df.png "style=" float: none; "Title=" screenshot from 2017-11-07 10-02-04.png "alt=" B99c82804f65301409d1e9083dc4a4df.png "/>
3. Configuration Server1
650) this.width=650; "src=" Https://s5.51cto.com/oss/201711/07/14bbb830023be24f556bc065832f8da7.png "style=" float: none; "Title=" screenshot from 2017-11-07 10-07-35.png "alt=" 14bbb830023be24f556bc065832f8da7.png "/>
650) this.width=650; "src=" Https://s1.51cto.com/oss/201711/07/8723fd5de21591ef0eeb3fd0773953a4.png "style=" float: none; "Title=" screenshot from 2017-11-07 10-05-50.png "alt=" 8723fd5de21591ef0eeb3fd0773953a4.png "/>
650) this.width=650; "src=" Https://s1.51cto.com/oss/201711/07/05c576cb5d79c83cf2ddd7743dfa8d5c.png "style=" float: none; "Title=" screenshot from 2017-11-07 10-06-11.png "alt=" 05c576cb5d79c83cf2ddd7743dfa8d5c.png "/>
650) this.width=650; "src=" Https://s1.51cto.com/oss/201711/07/918098c3a60656d77a1a639f80d40104.png "style=" float: none; "Title=" screenshot from 2017-11-07 10-06-35.png "alt=" 918098c3a60656d77a1a639f80d40104.png "/>
650) this.width=650; "src=" Https://s5.51cto.com/oss/201711/07/dcdbea9dbc5b3651ddb6fa5349bba73b.png "style=" float: none; "Title=" screenshot from 2017-11-07 10-06-54.png "alt=" Dcdbea9dbc5b3651ddb6fa5349bba73b.png "/>
4. Test is normal
650) this.width=650; "src=" Https://s4.51cto.com/oss/201711/07/2c7203832fa9a5c731055728b6ddd213.png "style=" float: none; "Title=" screenshot from 2017-11-07 10-07-50.png "alt=" 2c7203832fa9a5c731055728b6ddd213.png "/>
650) this.width=650; "src=" Https://s4.51cto.com/oss/201711/07/66799606294a10d0353b615696bc5ea2.png "style=" float: none; "Title=" screenshot from 2017-11-07 10-08-18.png "alt=" 66799606294a10d0353b615696bc5ea2.png "/>
Three. Install head plug-in, Web Access Management node, add data Storage node
1. Installing plugins
650) this.width=650; "src=" Https://s1.51cto.com/oss/201711/07/2cb1ec71f3850d532ae553a2ded79de1.png "style=" float: none; "Title=" screenshot from 2017-11-07 10-27-11.png "alt=" 2cb1ec71f3850d532ae553a2ded79de1.png "/>
650) this.width=650; "src=" Https://s1.51cto.com/oss/201711/07/f1ca6ab25d501c754dc1f75f323cb927.png "style=" float: none; "Title=" screenshot from 2017-11-07 10-29-17.png "alt=" F1ca6ab25d501c754dc1f75f323cb927.png "/>
To see if the plugin was successfully installed
650) this.width=650; "src=" Https://s4.51cto.com/oss/201711/07/28c5b92d507a764754e118bcd3463920.png "style=" float: none; "Title=" screenshot from 2017-11-07 10-29-24.png "alt=" 28c5b92d507a764754e118bcd3463920.png "/>
2. Web page by accessing the management node 172.25.29.1:9200/_plugin/head/
650) this.width=650; "src=" Https://s4.51cto.com/oss/201711/07/2e4cce9da8955fa209078ede38bc1544.png "style=" float: none; "Title=" screenshot from 2017-11-07 10-30-39.png "alt=" 2e4cce9da8955fa209078ede38bc1544.png "/>
Test acquisition
650) this.width=650; "src=" Https://s4.51cto.com/oss/201711/07/5330ba3251e7778cc355384f6ddb338c.png "style=" float: none; "Title=" screenshot from 2017-11-07 10-31-53.png "alt=" 5330ba3251e7778cc355384f6ddb338c.png "/>
3. Add a second host Server2
Install the Configuration service, note the Discovery option to open Server1 and Server2, and modify the correct host name
650) this.width=650; "src=" Https://s4.51cto.com/oss/201711/07/fb0d7b4420c84810e430ab6bca46f053.png "style=" float: none; "Title=" screenshot from 2017-11-07 10-34-05.png "alt=" Fb0d7b4420c84810e430ab6bca46f053.png "/>
650) this.width=650; "src=" Https://s4.51cto.com/oss/201711/07/564e78578636912ea311802167ec8eea.png "style=" float: none; "Title=" screenshot from 2017-11-07 10-32-32.png "alt=" 564e78578636912ea311802167ec8eea.png "/>
650) this.width=650; "src=" Https://s2.51cto.com/oss/201711/07/e1903fadded705349cad8f303d6395b5.png "style=" float: none; "Title=" screenshot from 2017-11-07 10-35-50.png "alt=" E1903fadded705349cad8f303d6395b5.png "/>
650) this.width=650; "src=" Https://s1.51cto.com/oss/201711/07/88aa213005bd23b652c55a7f9e5e6577.png "style=" float: none; "Title=" screenshot from 2017-11-07 10-36-32.png "alt=" 88aa213005bd23b652c55a7f9e5e6577.png "/>
Server1 and Server2 are going to change.
650) this.width=650; "src=" Https://s3.51cto.com/oss/201711/07/03208b391c48e0456669adc7c78b82ab.png "style=" float: none; "Title=" screenshot from 2017-11-07 10-36-59.png "alt=" 03208b391c48e0456669adc7c78b82ab.png "/>
650) this.width=650; "src=" Https://s3.51cto.com/oss/201711/07/396570ade29bcb17c75023c4dd9130d9.png "style=" float: none; "Title=" screenshot from 2017-11-07 10-37-24.png "alt=" 396570ade29bcb17c75023c4dd9130d9.png "/>
4. Access Master node, Server1 and server display normal
650) this.width=650; "src=" Https://s3.51cto.com/oss/201711/07/e11b5190428f6da447c005cd75313716.png "style=" float: none; "Title=" screenshot from 2017-11-07 10-37-34.png "alt=" E11b5190428f6da447c005cd75313716.png "/>
Enterprise-elk log Analysis for Linux