SOLR configuration Import MySQL data

1. Writing the configuration file

1) Write Data-config-comment.xml, which describes how to query MySQL data and how to import data into an index.

Suppose there is a database called MOOC, which has a table called comment, which represents the student's comments

which

• Entity corresponds to a row in the MySQL database table

• query for SQL queries for full library imports

• Queryimportquery SQL query corresponding to incremental import
  

• The deltaquery corresponds to an incremental import to get the latest modified row IDs, which are used in the meaning of Queryimportquery,sql

Date (updatetime) >= ' ${dih.last_index_time} ' OR date (writetime) >= ' ${dih.last_index_time}

represents the comment update time updatetime, or comment write time writetime than the last import time $ ( Dih.last_index_time) is also large.

<dataconfig>   <datasource type= "Jdbcdatasource"                 driver= "Com.mysql.jdbc.Driver"                url= "JDBC:MYSQL://LOCALHOST/MOOC"                 user= "Root"                 password= "Root"/>    <document>    <entity name= "comment"  query= "SELECT id,  Date_format (writetime,  '%y-%m-%dt%tz ')  as  ' writetime ',  title from comment " Deltaimportquery= "Select id, date_format (writetime,  '%y-%m-%dt%tz ')  as  ' Writetime ',  title from comment where id= ' ${dih.delta.id} ' "Deltaquery=" SELECT id FROM  comment where datE (updatetime)  >=  ' ${dih.last_index_time} '  or date (writetime)  >=  ' ${ Dih.last_index_time} ' ">          <field column=" ID " name=" id "/>     <field column=" writetime " name=" Writetime "/ >   <field column= "title"  name= "title"/>            </entity>     </document></dataconfig >

2) Suppose you want to create a SOLR kernel named Mooc, write fields in the Schema.xml file in its conf directory, add Id,writetime,title, where TEXT_CN needs to use the Chinese word breaker that I wrote on a blog.

<field name= "id" type= "string" indexed= "true" stored= "true" required= "true" multivalued= "false"/> <field name = "Writetime" type= "Tdate" indexed= "true" stored= "true"/><field name= "title" Type= "TEXT_CN" indexed= "true" Stored= "true"/>

3) Configuring SOLR's solrconfig.xml

In the D:\libs\solr-4.10.2\example\solr\mooc\conf directory, create the Data-config-comment.xml

Create a data import in Solrconfig.xml handler used to import the comment table, written as follows, where Data-config-comment.xml is the 1th step

<requesthandler name= "/dataimportcomment" class= "Org.apache.solr.handler.dataimport.DataImportHandler" > <lst name= "Defaults" > <str name= "config" >data-config-comment.xml</str> </lst> </ Requesthandler>

2. Configure the Java library files to use

Create D:\libs\solr-4.10.2\example\solr\mooc\lib, Copy solr-dataimporthandler-4.10.2 and Mysql-connector-java-5.1.26-bin to this, both libraries for importing and querying the database

3. Start SOLR

Enter the example directory of SOLR

Java-jar Start.jar

3. Import as index data

Run the following command in the browser to do a full library import, indicating that the data is imported into the SOLR kernel Mooc

Http://localhost:8983/solr/mooc/dataimportcomment?command=full-import&commit=y

If the Clean=false parameter is taken, the original data is not deleted

Incremental Import

Http://localhost:8983/solr/mooc/dataimportcomment?command=delta-import

4. Deleting an index file

Write an XML file with the contents of

<delete><query>*:* </query></delete>

Execute command

Java-durl=http://localhost:8983/solr/mooc/update-jar Post.jar Delete_docs.xml

5. Querying data

1) using SOLR edismax query, query "the" word, return id,title, and Writetime, in JSON format, can be changed to Wt=xml, representing the use of XML format, but for the convenience of program processing, with JSON

&wt=json&indent=true&defType=edismax&qf=title& in http://localhost:8983/solr/mooc/select?q= Omitheader=true&fl=id,title,writetime

2) streamlined, only check out the ID

Http://localhost:8983/solr/mooc/select?q= 's &wt=json&deftype=edismax&qf=title&omitheader=true &fl=id

Omitheader=true means to take out data information

3) Query date, using the following parameters

So far:

Fq=writetime:[* to now]

From August 19, 2011 to August 19, 2014:

fq=writetime:[2011-08-19t11:50:23z to 2014-08-19t11:50:23z]

From August 19, 2011 onwards:

fq=writetime:[2011-08-19t11:50:23z to *]

4) Highlight Keywords

http://localhost:8983/solr/mooc/select?q= Good &wt=json&indent=true&defType=edismax&qf=title& Omitheader=true&fl=id,title,writetime&hl=true

Hl=true indicates that highlighting is turned on

6. Front-end search operation process

The front-end Java script sends a request to the backend Servlet,servlet to the search engine for a query request, returning the result to the front end

7. Timing incremental Import data

1) Write the import script first sh_delta_import.sh

#!/bin/shcurl "Http://localhost:8983/solr/mooc/dataimportcomment?command=delta-import" Curl "http://localhost : 8983/solr/mooc/dataimportcomment?command=delta-import "

Again, because SOLR may return a warning for the first time and then import it once.

2) Configure/etc/crontab, add a line that represents 4 points per day

XX * * * root/mooc/solr-4.10.2/example/sh_delta_import.sh

3) Restart Timer program

/etc/rc.d/init.d/crond restart

SOLR configuration Import MySQL data