Coreseek configuration and incremental index merge index

: This article mainly introduces coreseek configuration and incremental index merge indexes. For more information about PHP tutorials, see. Guidance: I am a php Cainiao, and my company's business is not complicated, but recently I used full-text search, so I wanted to use sphinx.
There are roughly three parts: 1: installation; 2: 3: api call. This section describes how to configure and call APIs. I previously wrote a separate post to check the installation steps. if not, I can go to the official website. The installation steps are very clear.
I. Why should I use incremental indexes? In fact, I personally think that incremental indexing is completely unnecessary for businesses with small data volumes. You can re-generate the index on a regular basis. An incremental index is used to generate an index for the content added after the index generated last time. in this way, the data volume is small and does not affect business processing. then, indexes are merged regularly to maintain data uniformity, you also need to regularly generate new indexes.
1. the id of the last index generated must be recorded and can be stored in one table.
CREATE TABLETbl_pre_coursevideo (
Id int(11) NOT NULL DEFAULT '0',
Maxid int(11) NOT NULL DEFAULT '0',
PRIMARY KEY (Id )
) ENGINE=InnoDB DEFAULT CHARSET=utf8;

2. configuration file

Source mysql {type = mysql SQL _host = 127.0.0.1 SQL _user = root SQL _pass = 123456 SQL _db = test SQL _port = 3306 SQL _query_pre = SETNAMES utf8 SQL _query_pre = SETSESSION success = OFF # if an error is reported here, modify the mysql configuration file SQL _query_pre = REPLACEINTO tbl_pre_coursevideo SELECT1, MAX (id) FROM tbl_coursevideo # store the most IDs generated by the current index in the newly created table to prepare for the incremental index. SQL _query = SELECT id, title, create_time, subtitle, content, type FROM tbl_coursevideo WHERE id <= (SELECT maxid FROM tbl_pre_coursevideo WHERE id = 1) # The preceding SQL statement can be divided into two parts: before the WHERE statement, after the where statement is used to query data (based on your own business, is the filter SQL _attr_uint = ID for the maximum id of the record just now # The value read from SQL must be an integer SQL _attr_timestamp = create_time # The value read from SQL must be an integer, as the time attribute SQL _field_string = title # string field (full-text search is supported and the original text information can be returned) SQL _field_string = subtitle # string field (full-text search is supported and the original text can be returned) This information) SQL _field_string = content # string field (full-text search and original text information can be returned)} source increment: mysql {SQL _query_pre = SETNAMES utf8sql_query = SELECT id, title, create_time, subtitle, content, type FROMtbl_coursevideo WHERE id> (SELECT maxid FROM tbl_pre_coursevideo WHERE id = 1) # This is the incremental index data source SQL. Consistent with the above, the only change is that after the where condition, the query here is greater than the id of the last re-generated index, that is: just added data} # index definition index mysql {source = mysql # corresponding source name path =/usr/local/coreseek/var/data/mysql # Please change to actually used absolute path, example:/usr/local/coreseek/var /... docinfo = extern mlock = 0 morphology = none min_word_len = 1 html_strip = 0 # Chinese word segmentation configuration. for details, see: http://www.coreseek.cn/products-install/coreseek_mmseg/ Charset_dictpath =/usr/local/mmseg3/etc/# BSD, set in Linux environment,/symbol end # charset_dictpath = etc/# set in Windows environment,/symbol end, it is best to give an absolute path, for example: C:/usr/local/coreseek/etc /... charset_type = zh_cn.utf-8} index increment: mysql {source = increment path =/usr/local/coreseek/var/data/increment charset_dictpath =/usr/local/mmseg3/etc/charset_type = zh_cn.utf-8} # global index definition indexer {mem_limit = 128 M} # searchd service definition searchd {listen = 9312 read_timeout = 5 max_children = 30 max_matches = 1000 bytes = 0 preopen_indexes = 0 unlink_old = 1 pid_file =/usr/local/coreseek /var/log/searchd_mysql.pid # change it to the actual absolute path, example:/usr/local/coreseek/var /... log =/usr/local/coreseek/var/log/searchd_mysql.log # change it to the actual absolute path, for example,/usr/local/coreseek/var /... query_log =/usr/local/coreseek/var/log/query_mysql.log # change it to the actual absolute path, for example:/usr/local/coreseek/var /...}

Usage:
1. generate an index (the command below is my own installation environment path, and change it to my own)
/usr/local/coreseek/bin/indexer -c /usr/local/coreseek/etc/csft.conf --all# Generating indexes
In this case, a record is added to the tbl_pre_coursevideo table. It stores the largest id in your content table.

2,
/usr/local/coreseek/bin/searchd -c /usr/local/coreseek/etc/csft.conf# Enable background processes

At this time, if you search for the mysql data source, there is actually data.
3. incremental Index (prerequisite for adding data to the content table)
/usr/local/coreseek/bin/indexer -c /usr/local/coreseek/etc/csft.conf increment --rotate# After the execution is complete, several incremental index data entries are displayed, that is, the number of data entries you added to the content table. In this case, you can test whether your incremental index is successful without cl-> Query ($ keyword, 'credentials'). when you call an api, you can use the incremental index to Query the content you just added.
4. Merge indexes

Usr/local/coreseek/bin/indexer-c/usr/local/coreseek/etc/csft. conf
-Merge mysql increment-rotate

Cl-> Query ('query keyword ', 'mysql'); // you can Query the newly added data and previous data.

5. in order to ensure data consistency, you also need to regularly generate new indexes.

/Usr/local/coreseek/bin/indexer-c/usr/local/coreseek/etc/csft. conf -- all -- rotate # -- rotate can be added without affecting server search.

II,
1. call the api, which is encapsulated by myself. this class is included in the file after sphinxapi. php is installed.

Require 'sphinxapi. php'; class sphpointer {private $ host = '2017. 0.0.1 '; private $ port = 9312; private $ cl;/** @ desc constructor initializes the sphinx object */public function _ construct () {$ this-> cl = new SphinxClient (); $ this-> cl-> SetServer ($ this-> host, $ this-> port ); $ this-> cl-> SetConnectTimeout (1); $ this-> cl-> SetArrayResult (true); $ this-> cl-> SetMatchMode (SPH_MATCH_EXTENDED2 ); $ this-> cl-> SetRankingMode (SPH_RANK_WORDCOUNT );} /** @ desc search * @ param $ page number * @ param $ pagesize number of entries * @ param $ keyword search keyword * @ param $ source Index source */public function search ($ keyword, $ p, $ pagesize) {$ page = ($ P-1) * 10; $ this-> cl-> SetLimits ($ page, $ pagesize ); // pagination $ res = $ this-> cl-> Query ($ keyword, $ source); // sphsf-query }}

2. scheduled tasks
Yum install crontab // install
Crontab-e // open and edit
Then execute the incremental index regularly, merge the index, and then generate the new index.

Hold: If the index page is successfully merged. However, if the queried data is always empty, you can check the configuration file.

Path =/usr/local/coreseek/var/data/increment # In this configuration, whether the primary index is the same as the incremental index path, increment is the index file name, in the data folder.

So far. If there is nothing, take a good look at this blog, including comments, which are all my problems and comments. If you have any questions, you can leave me a message. Next time, let's talk about the word segmentation.

'). AddClass ('pre-numbering '). hide (); $ (this ). addClass ('Has-numbering '). parent (). append ($ numbering); for (I = 1; I <= lines; I ++) {$ numbering. append ($ ('

'). Text (I) ;}; $ numbering. fadeIn (1700) ;}) ;}; script

The above introduces coreseek configuration and incremental index merge indexes, including some content, and hope to be helpful to friends who are interested in PHP tutorials.