Elasticsearch River Import data from the database

Importing data using REIVER-JDBC in Elasticsearch2014-05-13 15:10 This site (3384)

Elasticsearch use REIVER-JDBC to import data, the need for friends can refer to the next.

The river module is provided in Elastisearch to fetch data from other data sources, which exists as a plug-in, and the existing river plug-ins include:

River Pluginsedit

1. Supported by Elasticsearch

• CouchDB River Plugin
• RabbitMQ River Plugin
• Twitter River Plugin
• Wikipedia River Plugin

2. Supported by the Community

• ActiveMQ River Plugin (by Dominik Dorn)
• Amazon SQS River Plugin (by Alex Bogdanovski)
• CSV River Plugin (by Martin Bednar)
• Dropbox River Plugin (by David Pilato)
• FileSystem River Plugin (by David Pilato)
• Git River Plugin (by Olivier Bazoud)
• GitHub River Plugin (by Ubervu)
• Hazelcast River Plugin (by Steve Samuel)
• JDBC River Plugin (by Jörg Prante)
• JMS River Plugin (by Steve Sarandos)
• Kafka River Plugin (by Endgame Inc.)
• LDAP River Plugin (by Tanguy Leroux)
• MongoDB River Plugin (by Richard Louapre)
• Neo4j River Plugin (by Steve Samuel)
• Open Archives Initiative (OAI) River Plugin (by Jörg Prante)
• Redis River Plugin (by Steve Samuel)
• RSS Plugin (by David Pilato)
• Sofa Plugin (by Adamlofts)
• SOLR River Plugin (by Luca Cavanna)
• St9 River Plugin (by Sunny Gleason)
• Subversion River Plugin (by Pascal Lombard)
• DynamoDB River Plugin (by Kevin Wang)

As can be seen, most of the data sources have been covered, especially for the relational database to provide a unified jdbc-river for data manipulation.

ELASTICSEARCH-RIVER-JDBC source code in: HTTPS://GITHUB.COM/JPRANTE/ELASTICSEARCH-RIVER-JDBC, the project provides detailed documentation, below the SQL Server as an example, simply explain how to use it.

First, you need to install ELASTICSEARCH-RIVER-JDBC and execute it under the Elasticsearch directory:

./bin/plugin --install jdbc --url http://xbib.org/repository/org/xbib/elasticsearch/plugin/elasticsearch-river-jdbc/1.1.0.1/elasticsearch-river-jdbc-1.1.0.1-plugin.zip

Then, install SQL Server's JDBC library, linked as: Microsoft
JDBC Driver. Copy the ' Sqljdbc4.jar ' into the Lib folder of the Elasticsearch installation directory.

Given the Elasticsearch cluster, the above two steps need to be performed on each node.

The final and most critical step is to build the river in Elasticsearch and let Elasticsearch automatically fetch data from SQL Server.

PUT/_river/mytest_river/_meta{    "type": "JDBC",    "jdbc": {        "driver": " Com.microsoft.sqlserver.jdbc.SQLServerDriver ",        " url ":" Jdbc:sqlserver://mysqlservername;databasename= Myproductdatabase ",        " user ":" admin "," password ":" Password ",        " SQL ":" Select ProductID as _id, CategoryID, Manufacturerid,mfname,producttitle,mfgpartnumber from Myproductstable (nolock),        "poll": "10m",        "strategy" : "Simple",            "index": "Myinventory", "        type": "Product",        "bulk_size": +,        "max_retries": 5,        " Max_retries_wait ":" 30s ","        max_bulk_requests ": 5,        " Bulk_flush_interval ":" 5s "    }}

The meaning of each parameter option is included in the documentation: Https://github.com/jprante/elasticsearch-river-jdbc/wiki/JDBC-River-parameters

Reference Documentation:

1. Http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/modules-plugins.html
2. http://blog.csdn.net/an74520/article/details/8740065
   
   
3. http://www.techovity.com/create-river-elasticsearch-ms-sql-server-automatic-data-transfer/

Elasticsearch River Import data from the database