Simple use of distributed search engine Elasticsearch

Official website: https://www.elastic.co/products/elasticsearch/

First, the characteristics

1. Support Chinese Word segmentation

2. Full-Text search engine supporting multiple data sources

3. Distributed

4. Lucene-based open source search engine

5. Restful API

Ii. Resources

• SMARTCN, default Chinese word breaker: HTTPS://GITHUB.COM/ELASTICSEARCH/ELASTICSEARCH-ANALYSIS-SMARTCN

• Mmseg:https://github.com/medcl/elasticsearch-analysis-mmseg

• Ik:https://github.com/medcl/elasticsearch-analysis-ik

• Pinyin, pinyin participle can be used to enter phonetic hints in Chinese: https://github.com/medcl/elasticsearch-analysis-pinyin

• Stconvert, Chinese Simplified and traditional interchange: Https://github.com/medcl/elasticsearch-analysis-stconvert

• Elasticsearch-servicewrapper:https://github.com/elasticsearch/elasticsearch-servicewrapper

• Elastic Hq,elasticsearch's monitoring tool: http://www.elastichq.org

• Elasticsearch-rtf:https://github.com/medcl/elasticsearch-rtf

Third, installation

• Server: Linux (CentOS 6.x)

• Java Environment: JDK 1.8.0

• elasticsearch:2.3.1

• ELASTICSEARCH-JDBC (data source plug-in): 2.3.1

• IK Analysis (Chinese word-breaker): 1.9.1

1. Install Java

Yum Install java-1.8.0

2, Installation Elasticsearch

#创建. Repo file (Elasticsearch.repo) Cat >>/etc/yum.repos.d/elasticsearch.repo << eof[elasticsearch-2.x]name =elasticsearch repository for 2.x packagesbaseurl=https://packages.elastic.co/elasticsearch/2.x/centosgpgcheck= 1gpgkey=https://packages.elastic.co/gpg-key-elasticsearchenabled=1eof# Import key:rpm--import https:// Packages.elastic.co/gpg-key-elasticsearchyum Install Elasticsearch

3. Create a Directory

Mkdir-p  /data/elasticsearch/datamkdir-p  /data/elasticsearch/logschown-r elasticsearch/data/ Elasticsearch/datachown-r Elasticsearch/data/elasticsearch/logs

4. Generate configuration file (/ETC/ELASTICSEARCH/ELASTICSEARCH.YML)

#集群名 (same cluster, name must be the same) cluster.name:my-application# service node name (different for each service node) node.name:node-1# data store path Path.data:/data/ elasticsearch/data# Service log path path.logs:/data/elasticsearch/logs# Service IP address network.host:0.0.0.0# service port http.port:9200

Iv. installation of IK 1. Installing MAVEN Tools

wget Http://repos.fedorapeople.org/repos/dchen/apache-maven/epel-apache-maven.repo-O/etc/yum.repos.d/ Epel-apache-maven.repoyum Install Apache-maven

2. Download the IK source package

git clone Https://github.com/medcl/elasticsearch-analysis-ik

3. Build the jar plug-in package

MVN cleanmvn compilemvn packageunzip target/releases/elasticsearch-analysis-ik-*.zipcp-r target/releases//usr/share /elasticsearch/plugins/ik

4. Configure thesaurus (IK comes with Sogou thesaurus)

Configuration:/usr/share/elasticsearch/plugins/ik/config/ik/ikanalyzer.cfg.xml

<entry key= "Ext_dict" >custom/mydict.dic;custom/single_word_low_freq.dic;custom/sougou.dic</entry>

Copy the jar package to the Elasticsearch plugins/analysis-ik directory, and then copy the extracted IK directory (config and dictionary) to the Elasticsearch config directory. Then edit the configuration file elasticsearch.yml and add a line later:

Index.analysis.analyzer.ik.type: "Ik"

Restartservice elasticsearch restart

Then enter the data and create the index

V. ELASTICSEARCH-JDBC1, using feeder method

wget http://xbib.org/repository/org/xbib/elasticsearch/importer/elasticsearch-jdbc/2.3.1.0/ Elasticsearch-jdbc-2.3.1.0-dist.zipunzip Elasticsearch-jdbc-2.3.1.0-dist.zip

Edit Data Import Script import.sh

Export jdbc_importer_home=/elasticsearch-jdbc-2.3.2.0bin= $JDBC _importer_home/binlib= $JDBC _importer_home/libecho ' {' type ': ' jdbc ', ' jdbc ': {' url ': ' Jdbc:mysql://127.0.0.1:3306/dbtest ', ' user ': ' Root ', ' password ': ' 123456 ', ' SQL ': ' SELECT * from TEST_TB "," index ":" Customer "," type ":" External "}} ' | Java     -cp "${lib}/*"     -dlog4j.configurationfile=${bin}/log4j2.xml     org.xbib.tools.Runner     Org.xbib.tools.JDBCImporter

Test

Curl ' localhost:9200/customer/external/_search?pretty&q=* '

2. Use river Mode

#安装elasticsearchcurl-ol https://download.elasticsearch.org/elasticsearch/elasticsearch/ ELASTICSEARCH-1.4.2.ZIPCD $ES _homeunzip path/to/elasticsearch-1.4.2.zip# Install the JDBC plugin./bin/plugin--install jdbc--url http://xbib.org/repository/org/xbib/elasticsearch/plugin/elasticsearch-river-jdbc/1.4.0.6/ elasticsearch-river-jdbc-1.4.0.6-plugin.zip# download mysql drivercurl-o mysql-connector-java-5.1.33.zip-l '/http dev.mysql.com/get/downloads/connector-j/mysql-connector-java-5.1.33.zip/from/http://cdn.mysql.com/' CP Mysql-connector-java-5.1.33-bin.jar $ES _home/plugins/jdbc/chmod 644 $ES _home/plugins/jdbc/* #启动elasticsearch./bin/ elasticsearch# stop Rivercurl-xdelete ' localhost:9200/_river/my_jdbc_river/'

JDBC Plug-in Parameters

curl-xput ' Localhost:9200/_river/my_jdbc_river/_meta '-d ' {"type": "JDBC", " JDBC ": {" url ":" Jdbc:mysql://localhost:3306/test "," User ":" "," Password ":" "," SQL ":" s Elect * FROM Orders "," index ":" Myindex "," type ":" MyType ", ...} '    If an array is passed to the JDBC field, multiple river sources are also available Curl-xput ' Localhost:9200/_river/my_jdbc_river/_meta '-d ' {<river parameters> "Type": "jdbc", "jdbc": [{<river definition 1>}, {<river definition 2>}]} ' cur L-xput ' Localhost:9200/_river/my_jdbc_river/_meta '-d ' {"type": "jdbc", "jdbc": {"Driver": "COM.MYSQ L.jdbc.driver "," url ":" Jdbc:mysql://localhost:3306/test "," User ":" root "," password ":" 123456 " , "SQL": "SELECT * from Test.student;", "Interval": "+", "index": "Test", "type": "St Udent "}} ' 

See if ES has synchronized this data

Curl-xget ' localhost:9200/test/student/_search?pretty&q=* '

Website address: https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-aliases.html

　

Reference

Https://www.elastic.co/guide/en/elasticsearch/guide/current/empty-search.html

Https://github.com/medcl/elasticsearch-analysis-ik

http://blog.csdn.net/clementad/article/details/46898013

Https://endymecy.gitbooks.io/elasticsearch-guide-chinese/content/elasticsearch-river-jdbc.html

Https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-aliases.html

Https://github.com/jprante/elasticsearch-jdbc

Http://www.voidcn.com/blog/wojiushiwo987/article/p-6058574.html

http://leotse90.com/2015/11/11/ElasticSearch%E4%B8%8EMySQL%E6%95%B0%E6%8D%AE%E5%90%8C%E6%AD%A5%E4%BB%A5%E5%8F% 8a%e4%bf%ae%e6%94%b9%e8%a1%a8%e7%bb%93%e6%9e%84/

http://www.jianshu.com/p/638ff7b848cc

Http://www.cnblogs.com/buzzlight/p/logstash_elasticsearch_kibana_log.html

Simple use of distributed search engine Elasticsearch