My Elasticsearch Cluster Deployment summary--Big Data search engine you have to know

Absrtact: There are three kinds of books in the World: 1, introduction of knowledge, 2, elaboration theory, 3, reference books; there are two kinds of knowledge in the world: 1, technology, 2, thought.here are myDeploymentElasticsearch clustering experience, they generally belong to the first category of knowledge "TechKnowledge (Technology)". But it also interspersed with some of my personal understanding. Please correct me.
keywords: ElasticSearch, search engine, cluster, big Data, SOLR, Big Data

three types of books and two types of knowledge:        Some books are an introduction to a new area of knowledge, cutting the field of knowledge from beginning to end, from the inside out, and absorbing the knowledge mainly in "Memory" (and "understanding"). and someA good book, often the whole book is in the elaboration of a sentence or a few words, one or several ideas, these are the most important part of the book, mainly in the "understanding." There are also some books that may be like tools, such as dictionaries, that have problems to check out, mainly in "find". There are also various types of books doped. These books have their own direction and focus, and the methods of reading are different.         for a variety of technologies, whether it is its installation steps, instructions or its problems, e-book, in the network of articles and FAQs will always have a solution, or technology it has unresolved issues, to fix the bug, but also most can find the answer on the Internet, (Can not find just waiting for you to invent and create, the Nobel Prize depends on you!) ）。 These are the "TechKnowledge" categories, like the first or third class, mainly in "memory" or "search". However, there are some ideological, cognitive things, online is difficult to obtain, it is not easy to express clearly, they should be categorized in "Thinking (thought)", like a second category of books, the focus is "understanding." While sharing "TechKnowledge", I would like to share these "thinking".          here are myDeploymentElasticsearch clustering experience, they generally belong to the first category of knowledge "TechKnowledge (Technology)". But it also interspersed with some of my personal understanding. Please correct me.  0. Environment configuration

Server: 3 units (Node1-NODE3)

Operating system: Cent OS 5.6, adding normal user deploy

Elasticsearch version: 1.7.1

JDK Version: 1.8

One, install JDK (all servers)

Install the JDK on each server (Node1-node3), and install the process slightly.

The following two to six steps can be selected to perform a server.

Second, installation Elasticsearch

1. Download elasticsearch-1.7.1.tar.gz

2, decompression: [[email protected] ~]$ TAR-XF elasticsearch-1.4.2.tar.gz

Third, install Elasticsearch-head plug-in

Elasticsearch-head is a elasticsearch cluster management tool, which is a standalone web program written entirely by HTML5.

1, the network environment, execute the following statement to install:

[Email protected] ~]$/elasticsearch-1.7.1/bin/plugin-install Mobz/elasticsearch-head

2. Offline Installation:

[Email protected] ~]$ Unzip Elasticsearch-head-master.zip

[Email protected] ~]$ mkdir./elasticsearch-1.7.1/plugins

[Email protected] ~]$ mkdir./elasticsearch-1.7.1/plugins/head

[Email protected] ~]$ mkdir./elasticsearch-1.7.1/plugins/head/_site

[Email protected] ~]$ cp-r ~/elasticsearch-head-master/*./elasticsearch-1.7.1/plugins/head/_site

In execution: [[email protected] ~]$./elasticsearch-1.7.1/bin/elasticsearch

After you start Elasticsearch, you can access://node1: 9200/_plugin/head/,

You can see such as:

Iv. installation of Elasticsearch-sql plug-in

The Elasticsearch-sql plugin can do this: Query elasticsearch using familiar SQL syntax. You can also use ES functions in SQL.

Execute the following statement to install:

[Email protected] ~]$/bin/plugin-u https://github.com/NLPchina/elasticsearch-sql/releases/download/1.3.5/ Elasticsearch-sql-1.3.5.zip--install SQL

After you start Elasticsearch, you can access:http://localhost:9200/_plugin/sql/,

You can see such as:

You can imagine what it's like to see this interface, right? More information can be obtained https://github.com/NLPchina/elasticsearch-sql, of course, not through the Elasticsearch-sql Elasticsearch when the relational database, But after all, Elasticsearch query statement is not as concise as SOLR, so it is convenient for a friend who is familiar with SQL to make SQL statements. The developer of this tool is really thoughtful.

Installation of Elasticsearch-bigdesk plug-in

Bigdesk is a cluster monitoring tool of Elasticsearch, which can be used to view the various states of ES cluster, such as CPU, memory usage, index data, search condition, HTTP connection number, etc.

1. Networked environment

[Email protected] ~]$/bin/plugin-install Lukas-vlcek/bigdesk

After you start Elasticsearch, you can access://Node1:9200/_plugin/bigdesk/,

You can see such as:

This plugin makes elasticsearch too intuitive, the data in which, how to change the display of the clear, great! Why does SOLR have no such stuff?

Vi. Installing the Elasticsearch-servicewrapper plug-in

The Elasticsearch-servicewrapper plugin is a service plugin for Elasticsearch.

After the plugin is downloaded Https://github.com/elasticsearch/elasticsearch-servicewrapper, unzip it. Copy the service directory to the bin directory of the Elasticsearch directory.

You can then install, start, and stop Elasticsearch by executing the following statement. (Do not do it first)

[[Email protected] service]# sh elasticsearch Install (must is root to perform this action.)

[[Email protected] service]# sh elasticsearch start

[[Email protected] service]# sh elasticsearch stop

This plugin is also practical, do not want to. Without this plugin, how many people still have to write a script boot up ah. Raise your hand if you need it!

VII. Copy the configured Elasticsearch to each server

After completing steps two to seven on a single server, get the final elasticsearch-1.7.1 folder. Package This folder and copy the SCP to each server.

Eight, configure Elasticsearch cluster

In each server (Node1, node3).

Execute the following statement to modify the contents of the configuration file elasticsearch.yml:

[Email protected] ~]$ vim elasticsearch-1.7.1/config/elasticsearch.yml

To modify some of the file contents after Elasticsearch.yml:

################################### Cluster ###################################

# Cluster Name identifies your Cluster for auto-discovery. If you ' re running

# Multiple clusters on the same network, make sure you ' re using unique names.

#

Cluster.name:elasticsearch_deploy

 

#################################### Node #####################################

# Node names is generated dynamically on startup, so you ' re relieved

# from configuring them manually. You can tie this node to a specific name:

#

Node.name: "node196"

Nine, start Elasticsearch cluster

You can start Elasticsearch by executing the SH elasticsearch start or./elasticsearch statement.

It can also be done by executing [[email protected] service]# sh elasticsearch install after: [[email protected] service]# chmod 777/home/deploy/ Elasticsearch-1.7.1/bin/service/elasticsearch, assigned to Elasticsearch Execute permissions. execute [[email protected] service]#/etc/init.d/elasticsearch start to start the Elasticsearch service. You can then execute [[email protected] service]# service Elasticsearch start to start the Elasticsearch. The system can also boot up after reboot.

After you have configured the same cluster.name on all servers (Node1-i) Elasticsearch, start elasticsearch on each server in turn, and you can view all the node states under the cluster by Bigdesk. End of cluster startup.

Ten, elasticsearch cluster function test

You can create new indexes on the head page, delete indexes, data browsing, queries, and so on, you need to set the number of shards, the number of replicas when creating a new index, and you can view shards and replicas on the Bigdesk page.

The data is stored in the Elasticsearch-1.7.1/data directory and is cluster.name for the storage of the catalog. Describes how two cluster.name are stored.

We monitor the Elasticsearch on the activated (Node1-node3). Figure 1, Figure 4 below, is the process of data replication movement within a cluster when a node in the cluster restarts. The number of shards in the index is 5, and the number of replicas is 1.

            

Figure 1: After three node stabilization                                                        Figure 2: Closing a node196

 

         

Figure 3: After two node stabilization                                              Figure 4: Start node196, restore three node stabilization

Can be through the above operation and image of the graph, see the outage a really little relationship to this cluster, data in the various servers freely flow, distribution, the server has no primary and secondary points.

SOLR I also used for a long time, I personally feel that elasticsearch than SOLR is good to have so many useful plug-ins can be used, that is, the image of the intuitive display, but also convenient and practical tools, thank you for these open source contributors, no you have not so a useful search engine Elasticsearch, many of the company's business will not be able to carry out! Of course , there are a lot of problems that need to be solved at present, Elasticsearch to talk later.

 Recent Big Data analysis I have a lot of sentiment ah. There are search engines, about distributed storage, and about distributed tasks. Have time to share with you. You are welcome to correct me! :) tomorrow is the first day after the 11 holiday! Come on, Angel Wang!

From Angel Wang 

My Elasticsearch Cluster Deployment summary--Big Data search engine you have to know