Tag: sql bin enter dcl ica master redis friendly lin
I. Overview
ELK official website https://www.elastic.co
ELK consists of three parts: Elasticsearch, Logstash and Kibana;
Elasticsearch is an open source distributed search engine. Its features are: distributed, zero configuration, automatic discovery, automatic index sharding, index copy mechanism, restful style interface, multiple data sources, automatic search load, etc.
Logstash is a completely open source tool that can collect, analyze, and store your logs for later use
Kibana is an open source and free tool that can provide log analysis friendly Web interface for Logstash and ElasticSearch, which can help you aggregate, analyze and search important data logs.
Common platform architecture
Simple architecture:
ELK + Redis
or
(Logstash / Flume)-> (Kafka / Redis)-> ES-> Web
It can be simpler:
Client (Filebeat)-> LogServer (Logstash-> Elasticsearch-> Kibana)
(You can directly output to ES without using Logstash and Filebeat)
Complex architecture:
Data collection layer [offline / real-time data, MySQL / ES / Logstash / Flume]
-> Data transmission layer [Kafka / Redis]
-> Data cache layer [Storm-> Kafka]
-> Compute storage layer [offline computing (Kylin / Spark SQL / Impala / MR), real time computing (Storm / Spark Streaming / Flink / Druid), storage (ES / MySQL / Hbase / HDFS / Kudu)]
-> Application layer [Web / REST API]
Run elasticsearch distributed log analysis and retrieval on windows
https://www.elastic.co/downloads
Download elasticsearch Logstash Kibana in the interface
1. Elasticsearch download, unzip zip, enter the bin file elasticsearch.bat
Visit http: // localhost: 9200
Successful start
2. Since elasticsearch is just string and other file information, you need to install the elasticsearch-head plugin
In the current version, since 6.4.0 is used to build the environment, you need to use tools such as node to install the plugin
Use grunt --version to view the version information of grunt installation
3. Modify the elasticsearch / conf folder Modify the elasticsearch.xml file,
Enter the config directory under the installation directory and modify the elasticsearch.yml file. Add at the end of the file to
http.cors.enabled: true
http.cors.allow-origin: "*"
node.master: true
node.data: true
Other operations
Then remove the comment of network.host: 192.168.0.1 and change to network.host: 0.0.0.0, remove the comment of cluster.name; node.name; http.port (that is, remove #)
Double-click elasticsearch.bat to restart es
4. Download the elasticsearch-head package
https://github.com/mobz/elasticsearch-head download the head plugin, choose to download zip
5. Unzip to the specified folder, G: \ elasticsearch-6.6.2 \ elasticsearch-head-master enter this folder, modify G: \ elasticsearch-6.6.2 \ elasticsearch-head-master \ Gruntfile.js in the corresponding Location plus hostname: '*'
6. After the modification is complete, use the cmd command to enter the elasticsearch-head-master folder, execute the command npm install and then execute npm run start to run the head
Successful operation Successful interface
Access interface
The above is the elasticsearch installation and viewing tutorial
Test elasticsearch-head
1. Use cmd request
curl -XGET http://127.0.0.1:9200/_cluster/health?pretty
Request result
2. Use elasticsearch-head to make requests
Request result
☆☆☆☆☆☆☆☆☆☆☆☆☆☆☆☆☆☆☆☆☆☆☆☆☆☆☆☆☆☆☆☆☆☆☆☆☆☆☆☆☆☆☆☆☆☆☆☆☆☆☆☆ 3.5 ☆☆
To start the logstash service, you need to modify the conf to add logstash.conf and paste it in the newly created file
input {
file {
type => "nginx_access"
path => "D: \ nginx \ logs \ access.log"
}
}
output {
elasticsearch {
hosts => ["192.168.10.105:9200"]
index => "access-% {+ YYYY.MM.dd}"
}
stdout {
codec => json_lines
}
}
Go to the bin folder and execute
Command 1 logstash.bat agent --f ../config/logstash.conf
Command 2 logstash.bat -f ../config/logstash.conf
Start logstash. If an error is reported, use "% CLASSPATH%" in logstash.bat in quotation marks, just execute
Startup error, cannot be regarded as continuous service
reason => "Expected one of #, input, filter, output at line 1, column 1 (byte1) after"}
Is a file format problem.
The default Windows file is utf8. Change the file format to: ansi format. The program can run normally bin \ logstash -f ./config/test.conf
Start Kibana service
Go to the kibana-6.4.0-windows-x86_64 / bin folder and double-click Kibana.bat
When you first start, you need to wait about one minute, the cmd command window does not respond
Elasticsearsh set Chinese word breaker
Use the installation command elasticsearch-plugin.bat install https://github.com/medcl/elasticsearch-analysis-ik/releases/download/v6.4.0/elasticsearch-analysis-ik-6.4.0.zip
Errors will be reported when using windows installation, it is better to download directly, and then unzip the file information to the elasticsearch / plugs / ik folder
Verification can be requested in kibana or in elasticsearsh
GET _search {"query": {"match": {"message": "中华"}}}
Set elasticsearsh as a windows system service
Execute elasticsearch-service.bat install in bin
Set logstash Kibana as a windows service
Install using nssm
The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion;
products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the
content of the page makes you feel confusing, please write us an email, we will handle the problem
within 5 days after receiving your email.
If you find any instances of plagiarism from the community, please send an email to:
info-contact@alibabacloud.com
and provide relevant evidence. A staff member will contact you within 5 working days.
