Mongo-connector integrated MongoDB to SOLR for incremental indexing

Mongo-connector Integrated MongoDB to SOLR Implementing incremental Indexing

Configuring a MongoDB Replication set

Reference: Deploying a replication set for testing and development

Installing Solr5.3

Reference: "Installing Solr5.3 under CentOS"

Installing Python2.7

Reference: "Installing Python2.7 under CentOS"

Install PIP

Reference: "Installing PIP under CentOS"

Installing Mongo-connector

method One: use Pip installation

Pip Install Mongo-connector

Installed to the default package directory for Ython:

/usr/local/lib/python2.7/site-packages

Method Two: Install as a service

1. Go to Https://github.com/mongodb-labs/mongo-connector/archive/master.zip to download mongo-connector-master.zip.

2. Unzip the entry directory.

Unzip MONGO-CONNECTOR-MASTER.ZIPCD Mongo-connector-master

3. Edit the configuration file.

VI Config.json

4. Install as a service.

Python setup.py Install_service

Create the Mongo-connector service under/ETC/INIT.D and copy the Config.json file to/etc/mongo-connector.json.

Uninstalling Mongo-connector Service

Python setup.py Uninstall_service

It will remove/etc/init.d/mongo-connector and/etc/mongo-connector.json

View service Status

Service Mongo-connector Status

Configure SOLR

The SOLR configuration file is available under the SOLR Data Catalog/data/solr/data/solr.xml

Create a core

Su-solr-c "/USR/LOCAL/SOLR/SOLR/BIN/SOLR create-c card-n data_driven_schema_configs"

Generated folder/data/solr/data/card, in the/data/solr/data/card/conf directory is the card configuration directory, you can configure synonyms, stop words.

Configure Solrconfig.xml

1. Make sure the Lukerequesthandler is enabled

The downstream application appears in the Solrconfig.xml file.

<requesthandler name= "/admin/luke" class= "Org.apache.solr.handler.admin.LukeRequestHandler"/>

2. Configure the hard commit (the frequency of flushing to the hard disk) and the soft commit (frequency committed to memory)

Configuring <autoCommit> and <autoSoftCommit> in the Solrconfig.xml file

<autocommit><maxtime>300000</maxtime><maxdocs>10000</maxdocs><opensearcher >true</openSearcher></autoCommit><!--Softautocommit is like autocommit except it causes a ' soft ' Commit which only ensures that changes be visiblebut does not ensure the data is synced to disk. This isfaster and the Near-realtime friendly than a hard commit.--><autosoftcommit><maxdocs>1000</ Maxdocs><maxtime>60000</maxtime></autosoftcommit>

Configure Schema.xml

1. Mongo Connector Storage metadata assists in handling rollbacks in each document. In order to support this data, you need to add the following information to your schema.xml:

<field name= "_ts" type= "Long" indexed= "true" stored= "true"/><field name= "ns" type= "string" indexed= "true" Stored= "true"/>

2. Configure configuration <uniqueKey>, <field> in Schema.xml

Start Mongo-connector

Method One: Start at the command line

Nohup sudo mongo-connector-m localhost:27019-t http://localhost:8983/solr/card-o oplog_progress.txt-n example.card-u _id-d Solr_doc_manager > Mongo-connector.out 2>&1

Method Two: Start with service

Service Mongo-connector Start

SOLR Delete all indexes

http://192.168.11.52:8983/solr/card/update/?stream.body=%3cdelete%3e%3cquery%3e*:*%3C/query%3E%3C/delete%3E &stream.contenttype=text/xml;charset=utf-8&commit=true

Reference:

Http://blog.mongodb.org/post/29127828146/introducing-mongo-connector

Https://github.com/mongodb-labs/mongo-connector/wiki/Installation

Https://github.com/mongodb-labs/mongo-connector/wiki/Usage-with-Solr

https://loutilities.wordpress.com/2012/11/26/complementing-mongodb-with-real-time-solr-search/

This article is from the SQL Server deep dives blog, so be sure to keep this source http://ultrasql.blog.51cto.com/9591438/1696083

Mongo-connector integrated MongoDB to SOLR for incremental indexing