I. Environmental preparedness
Role
SERVER IP
Logstash Agent
10.1.11.31
Logstash Agent
10.1.11.35
Logstash Agent
10.1.11.36
Logstash Central
10.1.11.13
Elasticsearch
10.1.11.13
Redis
10.1.11.13
Kibana
10.1.11.13
The architecture diagram is as follows:650) this.width=650; "src=" Http:
Due to project requirements, a set of ES services is required on the Linux platform. In the construction process, encountered a variety of problems. And then they all worked out. Now it is time to document the process, and the problems encountered, and how to resolve them.First, the Environment configuration
Operating system: Cent OS 7
Elasticsearch version: 1.4.1
JDK Version: 1.7
SSH Secure Shell version: 3.2.9
Original address: http://www.cnblogs.com/tianjixiaoying/p/4316011.htmlDue to project requirements, a set of ES services is required on the Linux platform. In the construction process, encountered a variety of problems. And then they all worked out. Now it is time to document the process, and the problems encountered, and how to resolve them.First, the Environment configuration
Operating system: Cent OS 7
Elasticsearch version: 1.4.1
J
Introduced
Elk is the industry standard log capture, storage index, display analysis System solutionLogstash provides flexible plug-ins to support a variety of input/outputMainstream use of Redis/kafka as a link between log/messageIf you have a Kafka environment, using Kafka is better than using RedisHere is one of the simplest configurations to make a note, Elastic's official website offers very rich documentationDo not use search engines to search, not much results, please directly reader Web
ElasticSearch, referred to as ES, is a lucene-based distributed full-text Search server, and SQL Server full-text index (fulltext index) a bit similar, are based on word segmentation and segmentation of the full-text search engine, with participle, synonym, stem query function , but ES inherently has distributed and real-time properties.One, install the Java SE EnvironmentInstall the Java JDK and configure the Java_home environment variables:1, downlo
Elasticsearch is a distributed, restful search and Analysis server, like Apache SOLR, which is a lucence-based index server, but I think the advantage of Elasticsearch versus SOLR is:
Lightweight: Easy to install, download the file after a command can be started;
Schema Free: You can submit JSON objects of any structure to the server, using Schema.xml to specify the index structure in SOLR;
Mul
Elasticsearch + Logstash + Kibana install X-Pack in the software package,Elasticsearch + Logstash + Kibana install X-Pack
X-Pack is an extension of an Elastic Stack that includes security, alarms, monitoring, reporting, graphics, and machine learning functions in an easy-to-install software package.1. install X-Pack in elasticsearch
Follow these steps to install
ElasticSearch 2 (9)-a summary of the story under ElasticSearch (a plot search)First top-down, after the bottom-up introduction of the elasticsearch of the bottom of the working principle, to try to answer the following questions:
Why doesn't my search *foo-bar* match foo-bar ?
Why do you add more files to compress indexes (index)?
Why does
Before you introduce the usage of Elasticsearch, let's talk about why you should use it. First of all to learn the search engine, certainly inevitably have heard LUCENE,SOLR and Elasticsearch are based on it. Spinx many articles, but the database is too intrusive (plug-in mode). Elasticsearch is one of the most popular distributed search engines of the moment. SO
First, IntroductionElasticsearch is an open source, distributed, restful search engine built on Lucene. Designed for cloud computing, it can achieve real-time search, stable, reliable, fast, easy to install and use. Supports data indexing using JSON with HTTP.Lucene is just a framework to take advantage of its functionality, to use Java, and to integrate Lucene into the program. To make things worse, Lucene is very complex and requires a lot of learning to understand how it works.Elasticsearch u
Fluentd is an open source collection event and log system that currently offers 150 + extensions that let you store big data for log searches, data analysis and storage.
Official address http://fluentd.org/plugin address http://fluentd.org/plugin/
Kibana is a Web UI tool that provides log analysis for ElasticSearch, and it can be used to efficiently search, visualize, analyze, and perform various operations on logs. Official Address http://www.elastic
Tutorial on setting up ElasticSearch in Window environment,
I. Prepare tools
1. JDK 1.8 and later development kits (no need to build them)
2. elasticsearch-6.0.0
3. elasticsearch-head-master
4. kibana-6.0.0
5. elasticsearch-analysis-ik-6.0.0 (temporarily unavailable)
2. Install El
Working Mechanism of ElasticSearch
Like Solr, ElasticSearch is an enterprise-level search engine that is based on Apache Lucene and has high reliability.
Some concepts in ElasticSearch correspond to relational databases. For example, a database is called an index in ES and a table is called a Type in ES.
The specific relationship is shown in the table below.
Rep
The system is Windows Server 2003.Elasticsearch is a stable, distributed, restful search engine based on Lucene. In fact, the so-called restful is that it provides a URL for you to call (indexing and retrieval), but it is too brutal to use it directly. Therefore, it also provides a series of client packages, which are equivalent to encapsulating the curl request, the languages supported by the client package include Java, PHP, Python, Ruby and Perl, a
field completion request URL (R ' ^search/ $ ', views.searchluoji,name= "search"), # Searching] 2, write the logical processing functionImplementing search data in logical processing functions(1) Get the user's search terms(2) using the native Elasticsearch (search engine) interface, to achieve the search, annotated:ELASTICSEARCH-DSL is on the original Elasticsearch to do the encapsulation
What is elasticsearch.
Elasticsearch is an open source search engine based on the Apache Lucene (TM). Whether in open source or proprietary domain, Lucene can be considered to be the most advanced, best performance, the most full-featured search engine library.
However, Lucene is just a library. To use it, you have to use Java as the development language and integrate it directly into your application, an
front-facing work
Current server is centos6.5+ 64bit
New Elasticsearch user, Es cannot start with root
useradd-d/usr/elasticsearch-g elastic-m Elasticsearch
Get Elasticsearch, this article with the v2.3.4 found to get the version es already have 5.x version, but spring data is currently only supported es2.x version, S
GitHub
"GitHub uses elasticsearch to search 20 TB of data, including 1.3 billion files and 130 billion lines of code"
I don't need to introduce this. coders understand it. GitHub upgraded their code search in January 2013 and changed it from SOLR to elasticsearch, the current cluster size is 26 index storage nodes and 8 client nodes (responsible for processing search requests). For details, see the official
outbound traffic is very scary, with a total traffic of MB in one minute. Check the two IP addresses, one in the United States and the other in Zhejiang Telecom;
Check the running process and find out the suspected process:
650) This. length = 650; "src =" http://s3.51cto.com/wyfs02/M02/42/73/wKioL1PXffLTYiecAAFKNFx1Tg0944.jpg "Title =" 004.png" width = "700" Height = "97" border = "0" hspace = "0" vspace = "0" style = "width: 700px; Height: 97px; "alt =" wkiol1pxffltyiecaafknfx1tg0944.jpg "/>
The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion;
products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the
content of the page makes you feel confusing, please write us an email, we will handle the problem
within 5 days after receiving your email.
If you find any instances of plagiarism from the community, please send an email to:
info-contact@alibabacloud.com
and provide relevant evidence. A staff member will contact you within 5 working days.