Mongo-connector integrates MongoDB to Solr for incremental Indexing

Mongo-connector integrates MongoDB to Solr for incremental Indexing

Configure the MongoDB replica set

Deploy a replica set for testing and development

Deployment replica set

1. create necessary data folders for each node:

1 mkdir-p/srv/mongodb/rs0-0/srv/mongodb/rs0-1/srv/mongodb/rs0-2

2. Run the following command to start the mongod instance:

First node:

Mongod -- port 27017 -- dbpath/srv/mongodb/rs0-0 -- replSet rs0 -- smallfiles -- oplogSize 128 -- logappend -- logpath/srv/mongodb/rs0-0/mongod. log &

The second node:

Mongod -- port 27018 -- dbpath/srv/mongodb/rs0-1 -- replSet rs0 -- smallfiles -- oplogSize 128 -- logappend -- logpath/srv/mongodb/rs0-1/mongod. log &

Third node:

Mongod -- port 27019 -- dbpath/srv/mongodb/rs0-2 -- replSet rs0 -- smallfiles -- oplogSize 128 -- logappend -- logpath/srv/mongodb/rs0-2/mongod. log &

3. You can use the following command to connect to the first instance:

Mongo -- port 27017

4. Use rs. initiate () in mongo to initialize the replica set. You can set the replica set configuration object in the following ways:

Rsconf = {
_ Id: "rs0 ",
Members :[
{
_ Id: 0,
Host: "}
]
}

5. Replace

Rs. initiate (rsconf)

6. View replica set configurations

Rs. conf ()

7. Use mongo to connect to primary and run the rs. add () command to add the second and third mongod instances to the replica set. Replace

Rs. add ("Rs. add ("

8. Run the rs. status () command to check the status of the replica set.

Install Solr5.3

Reference: Installing Solr5.3 in CentOS

Install Python2.7

Reference: Install Python2.7 in CentOS

Install pip

Reference: Installing pip under CentOS

Install mongo-connector

Method 1: Install using pip

1 pip install mongo-connector

Install it in the default ython package directory:

/Usr/local/lib/python2.7/site-packages

Method 2: Install as a service

1. Go to renewal.

2. decompress the package to the directory.

Unzip mongo-connector-master.zip
Cd mongo-connector-master

3. Edit the configuration file.

Vi config. json

4. Install as a service.

Python setup. py install_service

Create the mongo-connector service under/etc/init. d and copy the config. json file to/etc/mongo-connector.json.

Uninstall mongo-connector Service

Python setup. py uninstall_service

It will remove/etc/init. d/mongo-connector and/etc/mongo-connector.json

View service status

Service mongo-connector status

Configure Solr

The Solr configuration file solr. xml is available in the Solr data DIRECTORY/data/solr/data /.

Create a core

Su-solr-c "/usr/local/solr/bin/solr create-c card-n data_driven_schema_configs"

The folder/data/solr/data/card is generated. The configuration directory of the card is under the/data/solr/data/card/conf directory. You can configure synonyms and stop words.

Configure solrconfig. xml

1. Ensure that LukeRequestHandler is enabled

The following lines are applied to the solrconfig. xml file.

<RequestHandler name = "/admin/luke" class = "org. apache. solr. handler. admin. LukeRequestHandler"/>

2. Configure hard commit (the frequency of refreshing to the hard disk) and soft commit (the frequency of submitting to the memory)

Configure <autoCommit> and <autoSoftCommit> In the solrconfig. xml file.

<AutoCommit>
<Maxtime> 300000 </maxTime>
<Maxdoss> 10000 </maxDocs>
<OpenSearcher> true </openSearcher>
</AutoCommit>
<! -- SoftAutoCommit is like autoCommit doesn't it causes
'Soft 'commit which only ensures that changes are visible
But does not ensure that data is synced to disk. This is
Faster and more near-realtime friendly than a hard commit.
-->
<AutoSoftCommit>
<Maxdoss> 1000 </maxDocs>
<Maxtime> 60000 </maxTime>
</AutoSoftCommit>

Configure schema. xml

1. Mongo Connector stores metadata to help with rollback in each document. To support this data, you need to add the following information to your schema. xml:

<Field name = "_ ts" type = "long" indexed = "true" stored = "true"/>
<Field name = "ns" type = "string" indexed = "true" stored = "true"/>

2. Configure <uniqueKey> and <field> In schema. xml.

Start mongo-connector

Method 1: Start with a command line

Nohup sudo mongo-connector-m localhost: 27019-t http: // localhost: 8983/solr/card-o oplog_progress.txt-n example. card-u _ id-d solr_doc_manager> mongo-connector.out 2> & 1

Method 2: Start a service

Service mongo-connector start

Solr deletes all indexes

Http: // 192.168.11.52: 8983/solr/card/update /? Stream. body = % 3 Cdelete % 3E % 3 Cquery % 3E *: * % 3C/query % 3E % 3C/delete % 3E & stream. contentType = text/xml; charset = UTF-8 & commit = true

For more information about MongoDB, see the following links:

MongoDB 3.0 official version released and downloaded

CentOS compilation and installation of MongoDB

CentOS compilation and installation of php extensions for MongoDB and mongoDB

CentOS 6 install MongoDB and server configuration using yum

Install MongoDB2.4.3 in Ubuntu 13.04

MongoDB beginners must read (both concepts and practices)

MongoDB Installation Guide for Ubunu 14.04

MongoDB authoritative Guide (The Definitive Guide) in English [PDF]

Nagios monitoring MongoDB sharded cluster service practice

Build MongoDB Service Based on CentOS 6.5 Operating System

MongoDB details: click here
MongoDB: click here

This article permanently updates the link address: