Set up the SOLR environment and fully import MYSQL Data, and set up solr to import mysql

Set up the SOLR environment and fully import MYSQL Data, and set up solr to import mysql
SOLR Preface

Because solr is used in the project, it took more than a week to study solr. I will not talk about various problems, especially during the two days of research on timed incremental indexes, I don't know how many XXX programs are running. Well, I don't talk much about it. Let's share my research achievements with you. What else is SOLR, I cannot make it clear.

 

Version

SOLR

SOLR's latest version is 5.3, but I didn't use the latest version, using 4.10.4.

 

JDK

JDK version: jdk1.7.0 _ 13. Here we will talk about the fact that SOLR of version 4.10 requires JDK of version 1.7 and later. At the beginning, I used JDK of version 1.6, however, an error has been reported after deployment. After Baidu is replaced with JDK 1.7, it will be okay.

 

TOMCAT

The Tomcat version is apache-tomcat-7.0.63. This does not matter. You can also use tomcat 6.

 

 

Deployment

1. Download solr

Slave.

2. Extract

Decompress the downloaded solr4.10.4 package to a local folder. The unzipped folder is as follows:

 

3. Deployment

(1) Copy solr under the example \ webapps folder. war file to the webapps directory of tomcat, start tomcat to decompress solr. war, tomcat may report an error at this time, but you don't have to worry about it. decompress war (or manually decompress war), stop tomcat, and delete solr. war. Otherwise, it will be decompressed again each time it is started.

(2) back to the tomcat webapps directory, notepad open the solr \ WEB-INF \ web. xml file. Add the following code:

 

(3) As shown in the code above, you need to create a SOLR_HOME folder. You can decide which drive letter you want to put it on. My drive is F.

(4) After creating the SOLR_HOME folder, go back to the decompressed solr 4.10.4 folder and copy all the content under the example \ solr folder to the SOLR_HOME folder. After copying, The SOLR_HOME folder is as follows:

 

 

The dist and contrib folders are optional.

 

(5) Copy all jar files under the example \ lib \ ex folder to webapps \ solr \ WEB-INF \ lib.

(6) Run http: // localhost: 8080/solr and you will see the following page:

 

 

 

At this time, the SOLR environment is set up. (there is only one core. solr supports multicore. Do not talk about it for the time being, but we will talk about it later.) The following describes how to import data:

(1) Open F: \ SOLR_HOME \ collection1 \ conf and you can see solrconfig under this folder. open the file, and add the following code in the middle of <config> </config>:

 

 

(2) The data-config.xml file is introduced here, so you need to create a data-config.xml file in the same directory of solrconfig. xml, the content of the file is as follows:

 

 

Note:

A. The dataSource tag indicates the database of the table to which the index is to be created. You can check the configuration information and understand what you need. I will not talk about it much.

B .doc ument: entity indicates that there can be multiple entity tags in the document. One entity corresponds to one table. If it is an associated table, the entity tag must be nested in the entity tag, later, we will talk about it in a single table.

Name: indicates the name of the current entity, which is divided into multiple entity;

Pk: The primary key of the table corresponding to the current object;

Query: used for full indexing;

The next two are needed for incremental indexing. If it is just a full index, the next two are completely unnecessary;

 

C. field: the name of the field in the database. column corresponds to the name of the field in the database. You can set the name attribute by yourself, but it must correspond to the field in the schema. xml file;

 

Tuser table structure:

 

(3) schema. xml configuration

A. schema. xml file and data-config.xml are in the same directory, open the file, the configuration is as follows:

 

Note the field configuration. The name attribute corresponds to the name attribute in the field in the data-config.xml, and indexed is true to create an index for this field, if the value of stored is true, the content of this field is stored in the index file, which is easy to return during query. For fields with a large number of content, you only need to query but do not need to return the content, you can set stored to false;

B. Set the index ID.

Set the uniqueKey In the schema. xml file as the primary key in the Table, as shown below:

 

 

(4) elevate. xml

The elevate. xml file is also opened in the same directory of solrconfig. xml. The settings are as follows:

 

The reason for this setting is that the id in our table is of the int type, in schema. the IDS set in xml are also int type, but the solr index requires string type. If you do not comment out the preceding two <doc>, an error is returned.

 

(5) import the jar package

The configuration file for full import is the above three, but the import data from the database also needs three jar packages, one is the mysql Linked database jar package (mysql-connector-java-5.1.21.jar ), the other two are solr data import needs, is located in the decompressed solr 4.10.4 folder under the dist file under the solr-dataimporthandler-4.10.4.jar and solr-dataimporthandler-extras-4.10.4.jar, put these three files under tomcat \ webapps \ solr \ WEB-INF \ lib.

 

(6) start tomcat and access the localhost: 8080/solr path. Click the following location:

 

The following menu is displayed. Click Dataimport to display the following page:

 

Click the Execute button and click the refresh button below. The data import is completed when the data is changed to the following on the right:

 

Click the Query menu, enter the page, and click the Execute Query button to Query the index created by the imported data:

 

NumFound is the number of records in the database. The data in my database is as follows:

 

We can see that there are only 12 data records in my database, and the full index has been imported.

 

I will write it here for the time being. I will continue to write incremental import and regular incremental import next time. I shared a demo I made on Baidu cloud. If you need it, you can download it and put SOLR_HOME on drive F. If you don't want it on drive F, you can change the solr folder under the WEB-INF in web. in xml.

The address of Baidu online storage is http://pan.baidu.com/s/1sj9vlop. the content in the folder is as follows:

 

The Solr folder can be directly placed in the webapp under tomcat, SOLR_HOME. If you do not want to change the configuration, you can put it in the root directory of the F disk. If you want to put it on another disk, You need to modify the solr web. configuration in xml,

 

You don't need to talk about SQL files. As you all know, jar files are not used for the time being and can be used for regular incremental indexing.