Recently need to deal with hundreds of millions of data, the author in the recommendation of friends to learn elasticsearch, read many blogs on the internet also encountered a lot of problems,so I record the study and usethe process of elasticsearch. Elasticsearch Concept Online A lot, the author is not more verbose, directly into the actual combat.I. Environment
Objective
process, NIGNX format log into JSON, Logstash directly to Elasticsearch, and then through the Kibana GUI interface display analysis
Important NIGNX Log into JSON format, avoid nignx default log is a space, need a regular match, resulting in logstash too much CPUThe Elasticsearch machine configures the firewall, allowing only the specified Logstash machine accessKibana only listens for local 127.
Getting Started
Elasticsearch is a real-time, distributed search and analysis engine. It can help you deal with large-scale data at an unprecedented rate.It can be used for full-text search, structured search and analysis, and of course you can combine the three.
Wikipedia uses ElasticsearchFor full-text search and highlighting keywords, as well as providing search suggestions such as Search-as-you-type, Did-you-mean, and more. United Kingdom
The Gua
First, what is Elasticsearch?Elasticsearch is the first choice of open source full-text search engine, it can quickly store, search and analyze massive data. Stack Overflow,github, etc. are in use.Elasticsearch is written in Java and uses Lucene for indexing and searching internally, but it makes full-text retrieval simple, and instead provides a simple, consistent set of RESTful APIs by hiding the complexi
http://nkcoder.github.io/blog/20141106/elkr-log-platform-deploy-ha/
1. Architecture for highly available scenarios
In the previous article using Elasticsearch+logstash+kibana+redis to build a log management service describes the overall framework of log services and the deployment of various components, this article mainly discusses the Log service framework of high-availability scenarios, mainly from the following three aspects of consideration: As
A tool for real-time data synchronization between mongodb and ElasticSearch Based on netcore (ipv2es ),
Tools for real-time data synchronization between mongodb and ElasticSearch Based on netcore
One-to-one, one-to-many, multiple-to-one, and many-to-many data transmission modes are supported.
One-to-one-A mongodb collection corresponds to data synchronization between indexes of an
First, in a relatively simple way
Download the ZIP installation package First: Https://www.elastic.co/downloads/elasticsearch.
Unzip to the installation directory;
Enter the installation directory, enter the bin directory, start cmd;
Run Elasticsearch.bat. Error unsupported Major.minor version 52.0.
The reason is that I installed both jdk1.7 and jdk1.8. But the environment variable in my java_home set is jdk1.7,
://www.elastic.co/guide/en/elasticsearch/reference/current/setup.html first Download and install the public keyRPM--import Https://packages.elastic.co/GPG-KEY-elasticsearchConfigure the Yum sourceVim/etc/yum.repos.d/elasticsearch.repoelasticsearch.repoContent:[Elasticsearch-2.x]name=elasticsearch repository for 2.x packagesbaseurl= gpgcheck=1 gpgkey=http:// Packa
that you need to devote a lot of effort to the configuration to achieve a good presentation.Contents [Hide]
1 Basic Introduction
2 installation process
2.1 Preparation
2.2 Installing Java
2.3 Elasticsearch
2.4 Kibana
Basic IntroductionElasticsearch is currently the latest version of 1.7.1,Logstash is currently the latest version of 1.5.3Kibana is currently the latest version: 4.1.1Logstash forward
Elasticsearch is a distributed and extensible real-time search and analysis engine. It can help you search, analyze, and browse data, and often people don't anticipate the need for these features at the start of a project. Elasticsearch's appearance is to re-give the hard disk seemingly useless raw data new vitality.Elasticsearch each individual part is not a new creation. For example, full-text search has long been implemented, statistical systems an
River Machinery of ElasticsearchThe elasticsearch itself provides the river machinery for synchronizing data.Here, you can find the official recommended River now:http://www.elasticsearch.org/guide/en/elasticsearch/rivers/current/But the government did not provide HBase river.In fact, Es River is very easy, is a user packaged jar package, ES is responsible for finding a node. and start the river. Assuming n
1, Elasticsearch (search engine) queryElasticsearch is a very powerful search engine that uses it to quickly query to the required data.Enquiry Category:Basic query: Query with Elasticsearch built-in query criteriaCombine queries: Combine multiple query criteria together for compound queriesFiltering: Querying at the same time, filtering data with the filter condition without affecting the score2,
Because of the previous blog elasticsearch5.x on Linux distributed Installation (multi-node) installed Elasticsearch service is relatively new, so the elasticsearch will be made into a single service operation, need to use when routing forwarding request this service.
Because the Elasticsearch version is newer, the mileage version of spring data is used, and the
Recently, there is a business demand, that is, the Full-text search keyword query list, thus turning to the learning of ES, but also studied for most of the month, make a note, summed up their learning process.
Learning a new technology alone, it is always inevitable to go a lot of detours, this recommendation under the Basic course of ES, right, to study the official website tutorial on it.
1) Elasticsearch: Authoritative guide
Https://www.elastic.co
Installing Elasticsearch Head tool 1 on Linux, modifying the parameters of ElasticsearchEdit the Elasticsearch configuration file elasticsearch.yml/data/elasticsearch/elasticsearch-5.5.3/config/elasticsearch.ymlAdd the following configuration# 增加新的参数,这样head插件可以访问eshttp.cors.enabled: truehttp.cors.allow-origin: "*"Resta
1. Download the zip package of the elasticsearch-1.2.1, decompress it, and double-clickElasticsearch. bat(For Windows) Start the server (Listening to port 9200 by default)
Access http: // localhost: 9200. If the following JSON data is displayed, the startup is successful.
2. Sample Code for client connection:
Package COM. jiaoyiping. othersimple;/*** created with intellij idea. * User: Jiao Yiping * Date: 14-7-8 * Time: * to change this template us
Original link: http://jingyan.baidu.com/article/48206aead42b53216bd6b372.htmlElasticsearch is a Lucene-based search server. It provides a distributed multi-user-capable full-text search engine, based on a restful web interface.Recently work needed, and then studied the next es. By the way, the process of learning is recorded, it is also convenient for a long time to review again, or other environment to configure the reference.Tools/Materials
JDK, ES installation package
Installing
Because of the limited machine, this article only makes the cluster test of 3 nodes in a single machine.1. Cluster test informationElasticsearch version: elasticsearch-2.4.1Windowns version: Win102, decompression elasticsearch-2.4.1.zip to any directory, install Elasticsearch-head plug-in.Execute command in ${eshome}\bin directory: Plugin install Mobz/
Tags: elasticsearch search management plug-in Cluster 1. Install elasticsearch Elasticsearch is easy to install and can be decompressed immediately (you must install the Java environment in advance ). Download the latest version of elasticsearch running package from the http://www.elasticsearch.org; Three packages ar
Note: Elasticsearch version: 1.4.2First, installation and configuration1. Download Elasticsearch-analysis-ik-master.zip from Https://github.com/medcl/elasticsearch-analysis-ik2. Unzip the Elasticsearch-analysis-ik-master.zipUnzip Elasticsearch-analysis-ik-master.zip3. enter
The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion;
products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the
content of the page makes you feel confusing, please write us an email, we will handle the problem
within 5 days after receiving your email.
If you find any instances of plagiarism from the community, please send an email to:
info-contact@alibabacloud.com
and provide relevant evidence. A staff member will contact you within 5 working days.