ES Construction and simple operation

Successful interface diagram:

650) this.width=650; "Src=" Http://s5.51cto.com/wyfs02/M00/8B/05/wKioL1hCV_CyLhpWAAE_FX07Rm0881.png-wh_500x0-wm_3 -wmp_4-s_1993015982.png "title=" 2222.png "alt=" Wkiol1hcv_cylhpwaae_fx07rm0881.png-wh_50 "/>

Considerations for Using ES:

1, ES version 2.x above requires JDK 1.8 or more

2. Run ES cannot use root to run

3, ES directory must specify a normal user and group (authorization)

4, ES on memory and CPU consumption is higher

5, ES version in 5. x above system requires at least 2048 threads to be able to start, system version use CENTOS7 to

6, ES use the port to see open iptables:9200,9300, etc.

7, ES configuration other plug-ins to achieve visual monitoring of resources and other

8, ES version and plug-in version to match

9, ES cluster configuration, the first node to configure the SCP to other nodes can (modify the configuration file)

First, ES start

1. Download ES package. 2.4.2 version, unzip to/usr/local/and named Elasticsearch

2. Modify the configuration file

Cluster.name:my-application

Node.name:mycat

Path.data:/home/elastic/

Path.logs:/home/elastic/logs

network.host:192.168.1.114

http.port:9200

Discovery.zen.ping.unicast.hosts: ["192.168.1.114"]

3. Add Users

Useradd Elastic

4, to/usr/local/elasticsearch authorized

Chown-r Elastic.elastic/usr/local/elasticsearch

5. Switch User start Elastcisearch

Su Elastic

/usr/local/elasticsearch/bin/elasticsearch &

6. Verification Service

Login Http://192.168.1.114:9200/

{

"Name": "Mycat",

"Cluster_Name": "My-application",

"Cluster_uuid": "NWPLOAAEQRGRMXSQPL61SG",

"Version": {

"Number": "2.4.1",

"Build_hash": "C67dc32e24162035d18d6fe1e952c4cbcbe79d16",

"Build_timestamp": "2016-09-27t18:57:55z",

"Build_snapshot": false,

"Lucene_version": "5.5.2"

},

"Tagline": "Know, for Search"

}

Start ok~~!

8. Install kibana+sence Visual report

1, modify the Kibana configuration file/usr/local/kibana/config/kibana.yml

Service port used by Server.port:5601#kibana

Server.host: "192.168.1.114" #kibana实例主机

Elasticsearch.url: "http://192.168.1.114:9200" # #指向elasticsearch实例, monitor the status of the ES service, close the service first to close the Kibana service first

2. Start Kibana, this can be started in root mode

/usr/local/kibana/bin/kibana &

3. Loading Sence Plugin

/usr/local/kibana/bin/kibana plugin--install Elastic/sense

3, through the http://192.168.1.114:5601 can be visualized page

8, installation management and monitoring tools, using Marvel Plug-in, this plugin is charged, free trial for one months.

Marvel basic functionality is free but download is a full-featured version that requires subsequent application of the basic functionality of the license

Marvel is the Elasticsearch Management and monitoring tool

# #在/usr/local/elasticsearch/directory, install Marvel License and proxy, use ROOT to execute

Step 1:bin/plugin Install License

Bin/plugin Install Marvel-agent

#cd/usr/local/kibana/#把marvel服务安装到kibana里

Step 2:install Marvel into KibanaBin/kibana plugin--install elasticsearch/marvel/latest

#重启es和kibana服务

Step 3:bin/elasticsearch

Bin/kibana

Step 4:http://localhost:5601

Step 5: Configure index pattern

Modify the index name or the value in the pattern to:. marvel-* (The first time this configuration, post-use and index, write the index name can be)

650) this.width=650; "Src=" Http://s3.51cto.com/wyfs02/M00/8B/05/wKioL1hCWE6zXymCAADKRS9RPlU768.jpg-wh_500x0-wm_3 -wmp_4-s_566125820.jpg "title=" 11111.jpg "alt=" Wkiol1hcwe6zxymcaadkrs9rplu768.jpg-wh_50 "/>

You can now login to the Kibana page, you can see kibana\marvel\sence are integrated in the page also.

The default installation is a 30-day trial with a security module https://www.elastic.co/guide/en/marvel/current/license-management.html

If only the Marvel monitoring module needs to be replaced with the License basic License

Register for a Basic license

Installing license

Curl-xput-u admin ' http://

See if license was updated successfully

Curl-xget-u Admin:password ' Http://

Settings for full-text indexing:

1, download the package, unzip

Https://www.elastic.co/downloads/past-releases/logstash-2-4-1

2. Add Profile Directory

Mkdir/usr/local/logstash/conf

Touch/usr/local/logstash/conf/logstash.conf

Input {

JDBC {

Jdbc_driver_library = "/opt/logstash/mysql-connector-java-5.1.39/mysql-connector-java-5.1.39-bin.jar" # This package will be present on the Logstash server.

Jdbc_driver_class = "Com.mysql.jdbc.Driver"

jdbc_connection_string = "Jdbc:mysql://host:3306/database" #数据库信息

Jdbc_user = ""

Jdbc_password = ""

# schedule = "0 1 * * *"

statement = "SELECT * FROM AAA"

Clean_run = True

}

}

Output

{

Elasticsearch {

hosts = ["host:9200"]

index = "Index" #自定义

Document_type = "Index" #自定义

document_id = "%{id}" # # #设置ID为ES主键, when the database has an update operation, the original record will be overwritten

}

}

####

Input {

JDBC {

Jdbc_driver_library = "/usr/local/mysql-connector-java-5.1.40-bin.jar"

Jdbc_driver_class = "Com.mysql.jdbc.Driver"

jdbc_connection_string = "Jdbc:mysql://192.168.1.114:3306/test"

Jdbc_user = "Dlan"

Jdbc_password = "Root123"

Schedule = "0 1 * * *"

statement = "SELECT * FROM AAA"

jdbc_paging_enabled = "true"

Clean_run = "true"

}

}

Output

{

Elasticsearch {

hosts = ["192.168.1.114:9200"]

index = "Index"

Document_type = "Index"

document_id = "%{id}"

}

}

###

3. Start Logstash

/usr/local/logstash/bin/logstash-f/usr/local/logstash/conf/logstash.conf &

4, then kibana inside sense

Server localhost changed to es IP address

get/index_mycat/index_mycat/_search# return all the data

Get/index_mycat/index_mycat/_search

{

"Query": {

"term": {

"id": {

"Value": "7"

}

}

}

} #查看某个值

Get/index_mycat/index_mycat/_search

{

"Query": {

"Terms": {

"id": [

"1",

"2",

"3"

]

}

}

} #查询一个ID的多个值

Es Writing and querying data

Post/logstach20161203/testlog

{

"Date": "111111111",

"User": "Chen",

"MESG": "First Message"

}

GET _cat/indices

get/logstach20161203/testlog/avjcxeoovwwuc1n8avqu/_source# returns the original data, based on the value of the hash fetch

Get/logstach20161203/testlog/avjcxeoovwwuc1n8avqu?fields=user,mesg

#返回多个值

get/logstach20161203/testlog/_search/

{

"Query": {

"term": {

"User": {

"Value": "Chen"

}

}

}

}

How to update ES data:

One for full-amount commit, that is, to send a write request according to _ID

Post/logstach20161203/testlog/avjcxeoovwwuc1n8avqu

{

"Date": "111111112",

"User": "Chen",

"MESG": "First Message1"

}

The other is a partial update

Post/logstach20161203/testlog/avjcxeoovwwuc1n8avqu/_update

{"Doc": {"user": "AAA1A"}}

Curl-xpost ' http://192.168.1.114:9200/logstach20161203/testlog/AVjCxeOOVWWuc1n8AVQu/_update '-d ' {"Doc": {"User": " AAAA "}} '

Query for ES

Get/logstach20161203/testlog/_search?q=aaa1a

Get/logstach20161203/testlog/_search?q=user: "AAA1A"

Q= after writing is QueryString grammar.

1, full-text search: directly write the words searched, as in the above example?? First?;

2. Full-Text Search for single fields: Add field names and colons before searching for words, such as if you know the word

First? must appear in the MESG field, can write?? Mesg:first?;

3. Exact search of single field: double quotation marks before and after searching for words, such as?? User: "chenlin7"?;

4, a combination of multiple search conditions: can use?? Not?,?? and?? And?? OR?? To combine the search, note that it must be uppercase. Like what?? User: ("chenlin7"? OR? ") Chenlin ")? and? Not

Mesg:first?;

5. Does the field exist:? _exists_:user?? Do you have a request? User field exists,? _missing_:user?? Indicates a requirement? The user field does not exist;

6. Wildcard characters: use?? ??? Represents a single letter,? *?? Represents an arbitrary letter. Like what?? Fir?t?mess*?;

7. Regular: You need an expression that is more complex than a wildcard, and you can use regular. Like what

? Mesg:/mes{2}ages?/?. Note that the ES are poorly performing and the supported features are not

Especially powerful, try not to use. The regular syntax supported by ES?

See: https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-

Regexp-query.html#regexp-syntax;

8, approximate search: with?? ~?? Indicates that the search word may have one or two letters written wrong, please? ES? follow similar

9, the degree of return results. Like what?? Frist~?;

10, Range Search: The value and time, ES? can use range search, than

Such as:? rtt:>300, huh? date:["now-6h"? To? " Now "}?? such as of which,? []?? Represents an Endpoint

Values are included in the range,? {}?? Indicates that the endpoint value is not included in the range;

This article is from the "DBSpace" blog, so be sure to keep this source http://dbspace.blog.51cto.com/6873717/1879106

ES Construction and simple operation