Coreseek Chinese search

Brief Introduction: someone asked me recently, saying that mysql5.6 now supports full-text indexing of InnoDB, why is someone still using software such as SPhinX for MySQL databases.

First, many companies are still using mysql5.5. Full-text indexing is required for the InnoDB Storage engine.

Second: MySQL is not a Chinese-developed service, so it does not support Chinese word segmentation. This leads to the coreseek Chinese search to be explained next.

Because some children's shoes are confused about installation and basic use, write down my basic operations. If you have any questions, please leave a message.

Install

Stable version

wget http://219.232.239.243/uploads/csft/3.2/coreseek-3.2.14.tar.gzcurl -O -L http://mirrors.kernel.org/gnu/autoconf/autoconf-2.13.tar.gz

For earlier versions, the dependent package should also use earlier versions. If an error similar to must contain _ CV _ to be cached is reported later, this version is too high.

 

Test version

 wget http://219.232.239.243/uploads/csft/4.0/coreseek-4.1-beta.tar.gz

Dependency package

[[email protected] etc]# yum -y install gcc make gcc-c++ libtool autoconf automake imake mysql-devel libxml2-devel expat-devel[[email protected] mmseg-3.2.14]# history  |grep yum   58  yum -y install gcc make libiconv python   75  yum -y install gcc-c++  [[email protected]st src]# tar -xf coreseek-4.1-beta.tar.gz [[email protected] src]#curl -O -L http://mirrors.kernel.org/gnu/m4/m4-1.4.13.tar.gz [[email protected] src]# curl -O -L http://mirrors.kernel.org/gnu/autoconf/autoconf-2.65.tar.gz  [[email protected] src]# curl -O -L http://mirrors.kernel.org/gnu/automake/automake-1.11.tar.gz [[email protected] src]#  curl -O -L http://mirrors.kernel.org/gnu/libtool/libtool-2.2.6b.tar.gz

Compile and install all

./configure --prefix=/usr/local    make && make install    cd ..

[[email protected] bin]# iconv --version

Chinese Word Segmentation requires this support

 

Environment Modification

650) This. width = 650; "Title =" image 1.png "alt =" wKiom1PsTaGBL-aRAAGwxiAkeE0312.jpg "src =" http://s3.51cto.com/wyfs02/M02/45/D5/wKiom1PsTaGBL-aRAAGwxiAkeE0312.jpg "/>

[[Email protected] SRC] # localelang = en_US.UTF-8 [[email protected] SRC] # Vim/etc/sysconfig/i18n [[email protected] etc] # Cat/etc/sysconfig/ i18n # lang = "en_US.UTF-8" lang = "zh_CN.UTF-8" # sysfont = "latarcyrheb-sun16" sysfont = "latarcyrheb-sun16" [[email protected] ~] # Su-root (or restart the environment) [[email protected] SRC] # tar-XF coreseek-4.1-beta.tar.gz [[email protected] var] # cd/usr/local/src/coreseek-4.1-beta/testpack/var [[email protected] var] # Cat test/test. XML <? XML version = "1.0" encoding = "UTF-8"?> <Sphset: docset> <Sphinx: schema> <sphject: field name = "subject"/> <sphject: field name = "content"/> <Sphinx: ATTR name = "published" type = "timestamp"/> <Sphinx: ATTR name = "author_id" type = "int" bits = "16" default = "1"/> </Sphinx: schema> <Sphinx: document ID = "1"> <subject> the best fool maging news: Google's $30 billion acquisition of Baidu </subject> <published> 1270131607 </published> <content> according to foreign media reports, google will invest heavily in Baidu, involving up to $30 billion. Google used this to return to the Chinese mainland market. According to the report, Google and Baidu have now reached an acquisition agreement and will choose to announce its launch. Baidu's management team retains 100%, but will reduce the number of projects, including some malls and fengchao programs that are currently ineffective. The video website qiyi.com in the testing phase will enter more YouTube resources. (YouTube cannot be accessed in mainland China due to content review ).

See Chinese support

 

Compile and install

[[Email protected] SRC] # cd coreseek-4.1-beta/mmseg-3.2.14/[email protected] mmseg-3.2.14] #. /Bootstrap [[email protected] mmseg-3.2.14] #. /configure -- prefix =/usr/local/mmseg/[[email protected] mmseg-3.2.14] # Make & make install [[email protected] bin] #/usr/local/mmseg/ bin/mmseg-D/usr/local/mmseg/etc/[[email protected] coreseek-3.2.14] # cd csft-3.2.14/[[email protected] csft-3.2.14] # lsacinclude. m4 configure. AC install pymmseg sphinx-min.conf.inaclocal.m4 contrib libexpat python. m4 sphinx. specapi copying libstemmer_c smoke. sh sphinx. workspacebuildconf. sh csft.doc makefile. am sphinx03.sln srccodeblocks CSFT. pytest makefile. in sphinx05.sln testconfig Doc MISC sphinx08.sln winconfigure example. SQL mysqlse sphinx. conf. in [[email protected] csft-3.2.14] # sh buildconf. sh [[email protected] csft-4.1] #. /configure -- prefix =/usr/local/coreseek/-- without-Python -- without-unixodbc -- With-mmseg-eclipdes =/usr/local/mmseg/include/ mmseg/-- With-mmseg-libs =/usr/local/mmseg/lib/-- With-mysql [[[email protected] csft-4.1] # Make & make install so far the XML has been compiled complete

Support for MySQL

 

[[Email protected] csft-4.1] # Yum-y install mysql-devel libxml2-devel expat-devel make clean (clear previous compilation ). /configure -- prefix =/usr/local/coreseek -- without-unixodbc -- With-mmseg-separated des =/usr/local/mmseg/include/mmseg/--- mmseg-libs =/usr/local/mmseg/lib/-- With-mysqlmake & make install now the MySQL data source has been installed

Instance:

Use the default SQL statement and configuration file once.

mysql> GRANT ALL PRIVILEGES ON *.* TO ‘test‘@‘%‘ WITH GRANT OPTION;mysql> GRANT PROXY ON ‘‘@‘‘ TO ‘test‘@‘%‘ WITH GRANT OPTION; mysql>flush privileges; [[email protected] etc]# mysql -utest < example.sql [[email protected] etc]# pwd/usr/local/coreseek/etc [[email protected] etc]# cp sphinx.conf.dist csft.conf

[[email protected] etc]# vi  csft.conf

Open row 32nd
CSFT. conf
SQL _sock =/tmp/MySQL. Sock depends on the actual location of your MySQL
SQL _query_pre = set names utf8
Find SQL _query_info = select * from documents where id = $ ID and add SQL _query_info_pre = set names utf8 to the row (for Chinese display)

[[Email protected] etc] #/usr/local/coreseek/bin/indexer-C/usr/local/coreseek/etc/CSFT. conf -- allcoreseek Fulltext 4.1 [sph00002.0.2-Dev (r2922)] Copyright (c) 2007-2011, Beijing choice Software Technologies Inc (http://www.coreseek.com) using Config File '/usr/local/coreseek/etc/CSFT. conf '... indexing index 'test1 '... collected 4 docs, 0.0 mbsorted 0.0 mhits, 100.0% donetotal 4 docs, 193 bytestotal 0.007 sec, 24823 Bytes/sec, 514.46 docs/secindexing index 'test1stemmed '... collected 4 docs, 0.0 mbsorted 0.0 mhits, 100.0% donetotal 4 docs, 193 bytestotal 0.003 sec, 49411 Bytes/sec, 1024.06 docs/secskipping non-plain index 'did1 '... skipping non-plain index 'rt '... total 6 reads, 0.000 sec, 0.1 kb/call AVG, 0.0 msec/call avgtotal 18 writes, 0.000 sec, 0.1 kb/call AVG, 0.0 msec/call AVG [[email protected] data] # PWD/usr/local/coreseek/var/data [[email protected] data] # ls these files are generated by the above indexer command test1.spa test1.flac test1.spk test1.spp test1stemmed. spa test1stemmed.. SPK test1stemmed. spptest1.spd test1.spi test1.spm test1.sps test1stemmed. SPD test1stemmed. SPI test1stemmed. SPM test1stemmed. SPS

[[Email protected] data] #/usr/local/coreseek/bin/search test # You can find the table content in the defined database. 1. document = 1, Weight = 2421, group_id = 1, date_added = Wed Aug 13 01:29:11 2014 id = 1 group_id = 1 group_id2 = 5 date_added = 01:29:11 Title = test one content = This is my test document number one. also checking search within phrases.2. document = 2, Weight = 2421, group_id = 1, date_added = Wed Aug 13 01:29:11 2014 id = 2 group_id = 1 group_id2 = 6 date_added = 01:29:11 Title = test two content = This is my test document number two3. document = 4, weight = 1442, group_id = 2, date_added = Wed Aug 13 01:29:11 2014 id = 4 group_id = 2 group_id2 = 8 date_added = 2014-08-13 01:29:11 Title = Doc number four content = This is to test groupsmysql> Update test.doc uments set content =' grass mud horse '; [[email protected] etc] #/usr/local/coreseek/bin/search testcoreseek Fulltext 4.1 [sph1_2.0.2-Dev (r2922)] Copyright (c) 2007-2011, beijing choice Software Technologies Inc (http://www.coreseek.com) using Config File '/usr/local/coreseek/etc/CSFT. conf '... index 'test1': query' test': returned 3 matches of 3 Total in 0.000 sec displaying matches: 1. document = 1, Weight = 2421, group_id = 1, date_added = Wed Aug 13 01:29:11 2014 id = 1 group_id = 1 group_id2 = 5 date_added = 01:29:11 Title = test one content = straw horse 2. document = 2, Weight = 2421, group_id = 1, date_added = Wed Aug 13 01:29:11 2014 id = 2 group_id = 1 group_id2 = 6 date_added = 01:29:11 Title = test two content = straw horse 3. document = 4, Weight = 1442, group_id = 2, date_added = Wed Aug 13 01:29:11 2014 id = 4 group_id = 2 group_id2 = 8 date_added = 01:29:11 Title = Doc number four content = straw horse

Detailed use of the method reference: http://www.coreseek.cn/

This article from the "smile ah" blog, please be sure to keep this source http://zuoyuezong.blog.51cto.com/6444039/1539889