Sphinx + SCWS + Mysql + PHP Full Text Search

Source: Internet
Author: User
Tags bz2 fpm install php

1 Install php,mysql before installing

php5.5.29 installation: http://blog.csdn.net/clevercode/article/details/52198698.

mysql-5.5.27:http://blog.csdn.net/clevercode/article/details/45499231.


2 Downloads

2.1 This installation of the resource bundle, configuration files, test files

http://download.csdn.net/detail/clevercode/9602949.


2.2 Other

sphinx:http://sphinxsearch.com/downloads/release/
Sphinx PHP Extension: Http://pecl.php.net/package/sphinx
scws:http://www.xunsearch.com/scws/download.php
SCWS Thesaurus: http://www.xunsearch.com/scws/down/scws-dict-chs-utf8.tar.bz2


3 Other References

"Sphinx Scws Full-Text Search installation configuration detailed": Http://blog.51yip.com/mysql/1659.html.
"PHP based on SPHINX+SWCS Chinese word segmentation of the full text search": http://www.bubuko.com/infodetail-1024880.html.
Sphinx Inverted Table Introduction ("SPH", "Spa", "SPI", "SPD", "SPP", "SPM"): http://blog.csdn.net/todaylxp/article/details/4269223.

"Sphinx configuration File Full resolution": http://www.cnblogs.com/yjf512/p/3598332.html.

4 Installation

4.1 Installing Sphinx

# mkdir-p/usr/local/src/sphinx# Cd/usr/local/src/sphinx # tar zxvf sphinx-2.2.5-release.tar.gz   # CD Sphinx-2.2.5-r Elease   #/configure--prefix=/usr/local/sphinx2--with-mysql=/usr/local/mysql   

4.2 Installing the Sphinx Client

# cd/usr/local/src/sphinx/sphinx-2.2.5-release/api/libsphinxclient   //sphinx-2.2.5-release Directory   #./ Configure--prefix=/usr/local/sphinx2/libsphinxclient   # make && make install  

4.3 Installing Sphinx PHP Extensions

(Phpize in/usr/local/php5/bin/phpize, if there is no phpize command, ln-s to/usr/bin)

# cd/usr/local/src/sphinx# Tar zxvf sphinx-1.3.1.tgz   # cd sphinx-1.3.1   # phpize   

4.4 Installing SCWS

# cd/usr/local/src/sphinx# tar xvjf scws-1.2.3.tar.bz2# mkdir/usr/local/scws   # cd scws-1.2.3   #./configure--PR efix=/usr/local/scws/   

4.5 Installing SCWS PHP Extensions

# cd/usr/local/src/sphinx/scws-1.2.3/phpext  # Phpize   

4.6 Installing the SCWS thesaurus

# tar xvjf scws-dict-chs-utf8.tar.bz2-c/usr/local/scws/etc/# www for php-fpm run user # Chown www:www/usr/local/scws/etc/ Dict.utf8.xdb

5 Creating a MySQL data source

5.1 Test.sql

/*navicat MySQL Data transfersource server:192.168.101.130source Server version:50527source host:1         92.168.101.130:3306source database:testtarget Server type:mysqltarget server Version:50527file Encoding : 65001date:2016-08-14 13:46:48*/set foreign_key_checks=0;--------------------------------Table structure for "use Rinfo '------------------------------DROP TABLE IF EXISTS ' userinfo ';   CREATE TABLE ' userinfo ' (' id ' int (one) unsigned not null auto_increment, ' userid ' int (one) unsigned not null DEFAULT ' 0 ', ' Addtime ' datetime NOT NULL, ' post ' varchar (no null DEFAULT ' ', ' summary ' text NOT NULL, PRIMARY KEY (' id ')) ENG Ine=innodb auto_increment=21 DEFAULT Charset=utf8;--------------------------------Records of userinfo-------------- ----------------insert INTO ' userinfo ' values (' 17 ', ' 1 ', ' 2012-06-01 00:24:54 ', ' Hung ', ' Olympic '); insert INTO ' userinfo ' values (' 18 ', ' 2 ', ' 2014-08-19 10:24:54 ', ' Chinese ', ' primitive '); INSERT into ' userinfo ' VALUES (' 19 ', ' 3 ', ' 2015-08-01 12:24:54 ', ' primitive ', ' nimne '); INSERT into ' userinfo ' VALUES (' 20 ', ' 4 ', ' 2013-08-01 00:24:54 ', ' The force of the primitive ', ' 23 gold ');--------------------------------table structure for ' users '------------------------------DROP table IF EXISTS ' Users '; CREATE TABLE ' users ' (' id ' int (one) unsigned not null auto_increment, ' username ' varchar "not null DEFAULT ', PRIMA RY KEY (' id ')) engine=innodb auto_increment=5 DEFAULT Charset=utf8;--------------------------------Records of users--- ---------------------------insert INTO ' users ' values (' 1 ', ' Zhangsan '); insert into ' users ' values (' 2 ', ' Zhangsi '); I Nsert into ' users ' values (' 3 ', ' John Doe '); INSERT into ' users ' values (' 4 ', ' Harry ');

5.2 Executing the source Test.sql



5.3 Data sources



6 Configuring Sphinx

The following 3 configurations of Searchd are important. Mysql-h 127.0.0.1-p 9306 access. 9312 can be accessed for PHP. Query_log_format Record The format of the log.

1 # PHP
Listen = 9312
2 # MySQL
Listen = 9306:mysql41

3 # Log_format

Query_log_format = SPHINXQL

# Vim /usr/local/sphinx2/etc/ sphinx.conf

SOURCE users{type= mysqlsql_host= 127.0.0.1sql_user= rootsql_pass= 123456sql_db= testsql_port= 3306# Optional, default  is 3306sql_query_pre = set NAMES UTF8 sql_query_pre = set SESSION Query_cache_type=off sql_query = SELECT a.ID, A.userid,b.username, Unix_timestamp (a.addtime) as Addtime, A.post, a.summary from UserInfo a left join users B on a . UserID = b.id Sql_attr_uint = UserID Sql_field_string = Username Sql_field_string = Post Sql_attr_timestamp = Addtime Sql_ranged_throttle = 0 #sql_attr_uint = group_id#sql_attr_timestamp= date_added#sql_ranged_throttle= 0}source src1throttled:users {sql_ranged_throttle= 100}index users {Source = users path =/usr/local/sphinx2/var/data/users doc info = extern Mlock = 0 morphology = None Min_word_len = 1 Html_strip = 1 charset_table = u+ff10. u+ff19->0..9, 0..9, u+ff41. U+ff5a->a. Z, U+FF21. U+ff3a->a. Z,a. Z->a. Z, A.. Z, u+0149, u+017f, u+0138, U+00DF, U+00FF, u+00c0. U+00d6->u+00e0. U+00f6,u+00e0. u+00F6, U+00d8. U+00de->u+00f8. U+00fe, U+00f8. U+00fe, u+0100->u+0101, u+0101,u+0102->u+0103, u+0103, u+0104->u+0105, u+0105, U+0106->U+0107, U+0107, U+ 0108->u+0109,u+0109, u+010a->u+010b, u+010b, u+010c->u+010d, u+010d, u+010e->u+010f, U+010F,U+0110->U +0111, u+0111, u+0112->u+0113, u+0113, u+0114->u+0115, u+0115, u+0116->u+0117,u+0117, U+0118->U+0119, U+ 0119, u+011a->u+011b, u+011b, u+011c->u+011d, u+011d,u+011e->u+011f, u+011f, u+0130->u+0131, U+0131, U+ 0132->u+0133, u+0133, u+0134->u+0135,u+0135, u+0136->u+0137, u+0137, u+0139->u+013a, U+013A, U+013B->U +013c, u+013c,u+013d->u+013e, u+013e, u+013f->u+0140, u+0140, u+0141->u+0142, U+0142, U+0143->U+0144,U+ 0144, u+0145->u+0146, u+0146, u+0147->u+0148, u+0148, u+014a->u+014b, u+014b,u+014c->u+014d, U+014D, U+ 014e->u+014f, u+014f, u+0150->u+0151, u+0151, u+0152->u+0153,u+0153, u+0154->u+0155, U+0155, U+0156->U +0157, u+0157, U+0158-> u+0159, u+0159,u+015a->u+015b, u+015b, u+015c->u+015d, u+015d, u+015e->u+015f, U+015F, U+0160->U+0161,U+ 0161, u+0162->u+0163, u+0163, u+0164->u+0165, u+0165, u+0166->u+0167, u+0167,u+0168->u+0169, U+0169, U+ 016a->u+016b, u+016b, u+016c->u+016d, u+016d, u+016e->u+016f,u+016f, u+0170->u+0171, U+0171, U+0172->U +0173, u+0173, u+0174->u+0175, u+0175,u+0176->u+0177, u+0177, u+0178->u+00ff, u+00ff, U+0179->U+017A, U+ 017A, u+017b->u+017c,u+017c, u+017d->u+017e, u+017e, u+0410..u+042f->u+0430..u+044f, U+0430..U+044F,U+05D0 .. U+05ea, u+0531..u+0556->u+0561..u+0586, u+0561..u+0587, u+0621..u+063a, U+01B9,U+01BF, U+0640..U+064A, U+0660..U+ 0669, u+066e, u+066f, U+0671..u+06d3, u+06f0. u+06ff,u+0904..u+0939, u+0958..u+095f, u+0960..u+0963, u+0966..u+096f, u+097b. U+097F,U+0985..U+09B9, U+09ce, U+09DC. U+09e3, U+09e6. U+09ef, U+0a05. U+0a39, U+0a59. U+0a5e,u+0a66. u+0a6f, U+0a85. U+0AB9, U+0ae0. U+0ae3, U+0ae6. U+0AEF, U+0b05. u+0b39,u+0b5C.. U+0b61, u+0b66. u+0b6f, U+0b71, u+0b85. U+0BB9, U+0be6. U+0BF2, U+0C05. U+0c39,u+0c66. u+0c6f, u+0c85. U+0CB9, U+0CDE. U+0ce3, U+0ce6. U+0CEF, u+0d05. U+0d39, U+0d60,u+0d61, U+0d66. u+0d6f, u+0d85. U+0DC6, u+1900..u+1938, u+1946..u+194f, u+a800. u+a805,u+a807. u+a822, U+0386->U+03B1, u+03ac->u+03b1, U+0388->u+03b5, U+03ad->u+03b5,u+0389->u+03b7, U+03AE->U+ 03b7, U+038A->U+03B9, U+0390->U+03B9, U+03AA->U+03B9,U+03AF->U+03B9, U+03CA->U+03B9, U+038C->U+ 03BF, U+03CC->U+03BF, U+038e->u+03c5,u+03ab->u+03c5, U+03b0->u+03c5, U+03CB->U+03C5, U+03CD->U+ 03C5, U+038F->U+03C9,U+03CE->U+03C9, U+03C2->U+03C3, u+0391..u+03a1->u+03b1. U+03c1,u+03a3. U+03a9->u+03c3. U+03C9, u+03b1. U+03C1, U+03C3. U+03C9, U+0e01. U+0e2e,u+0e30. U+0E3A, U+0e40. U+0e45, U+0e47, U+0e50. U+0e59, u+a000. u+a48f, u+4e00. U+9FBF,U+3400..U+4DBF, U+20000..U+2A6DF, u+f900. U+faff, u+2f800. U+2FA1F, U+2e80. U+2eff,u+2f00. U+2FDF, u+3100..u+312f, u+31a0. U+31bf, u+3040..u+309f, u+30a0. U+30ff,u+31f0. U+31FF, u+ac00. U+D7AF, u+1100..u+11ff, u+3130..u+318f, u+a000. u+a48f,u+a490. U+A4CF Ngram_len = 1 Ngram_chars = u+4e00. U+9FBF, U+3400..U+4DBF, U+20000..U+2A6DF, u+f900. u+faff,u+2f800. U+2FA1F, U+2e80. U+2eff, u+2f00. U+2FDF, u+3100..u+312f, u+31a0. u+31bf,u+3040..u+309f, u+30a0. U+30ff,u+31f0. U+31FF, u+ac00. U+D7AF, u+1100..u+11ff,u+3130..u+318f, u+a000. u+a48f, u+a490. U+A4CF}common{}indexer{mem_limit= 128m}searchd{#phplisten = 9312#mysql Listen = 9306:mysql41log=/u sr/local/sphinx2/var/log/searchd.logquery_log=/usr/local/sphinx2/var/log/query.logquery_log_format = sphinxqlread_timeout= 5client_timeout= 300max_children= 30persistent_connections_limit= 30pid_file=/usr/local/ sphinx2/var/log/searchd.pidseamless_rotate= 1preopen_indexes= 1unlink_old= 1mva_updates_pool= 1Mmax_packet_size= 8mmax_filters= 256max_filter_values= 4096max_batch_queries= 32workers= Threads # for RT to work}

7 Configuring PHP.ini

# Vim/etc/php.ini Add

[sphinx]extension = sphinx.so[scws]extension = Scws.soscws.default.charset = Utf-8scws.default.fpath =/usr/local/scws /etc

8 Start sphinx,php-fpm8.1, start Sphinx

# pkill searchd#/usr/local/sphinx2/bin/indexer--config/usr/local/sphinx2/etc/sphinx.conf--all  #/usr/local/ Sphinx2/bin/searchd--config/usr/local/sphinx2/etc/sphinx.conf

# if the error libmysqlclient.so.18.
# ln-s/usr/local/mysql/lib/libmysqlclient.so.18/usr/lib/libmysqlclient.so.18 #32机器
# ln-s/usr/local/mysql/lib/libmysqlclient.so.18/usr/lib64/libmysqlclient.so.18 #64位机器


8.2, restart PHP-FPM (if the Web calls PHP)
Restart PHP-FPM

9 Test Sphinx Full Text Search9.1 Client Testing






9.2 PHP Test

test.php

<?php header ("content-type:text/html; Charset=utf-8 "); $b _time = Microtime (true); $key = "The force of the primitive"; $index = "Users"; ======================================== participle $so = scws_new (); $so->set_charset (' utf-8 '); The default thesaurus $so->add_dict (ini_get (' Scws.default.fpath '). '/dict.utf8.xdb '); Custom Thesaurus//$so->add_dict ('./dd.txt ', scws_xdict_txt); The default rule $so->set_rule (Ini_get (' Scws.default.fpath '). '/rules.utf8.ini '); Sets whether to remove some special punctuation marks when the word breaker returns the result $so->set_ignore (true); Set whether the word breaker returns the result of a duplex split, such as "Chinese" return "Chinese + people + Chinese" three words. Bitwise XOR 1 | 2 | 4 | 8 distinction: Short Words | binary | Main Word | All word//1,2,4,8 correspond to constants Scws_multi_short scws_multi_duality scws_multi_zmain Scws_multi_zall $so->set_multi (false); Set whether the idle text is automatically aggregated by Word segmentation $so->set_duality (FALSE); Set search terms $so->send_text ($key); $words _array = $so->get_result (); $words = ""; foreach ($words _array as $v) {$words = $words. ' | ('. $v [' word ']. ') '; }//Join Whole Word # $words = ' ('. $key. ') '. $words; $words = Trim ($words, ' | '); $so->close (); Echo ' <p> input: '. $key. ' </p> '. ' \ r \ n"; Echo ' <p> participle: '. $words. ' </p> '. ' \ r \ n ";//======================================== search $SC = new sphinxclient (); $SC->setserver (' 127.0.0.1 ', 9312); # $SC->setmatchmode (Sph_match_all); $SC->setmatchmode (sph_match_extended); $SC->setarrayresult (TRUE); $res = $sc->query ($words, $index); Print_r ($res); $e _time = Microtime (true); $time = $e _time-$b _time; Exit;? >


3) Printing Results

From the words results, you can see the number of articles hit after a dollar participle.

<p> input: The force of the Primitive </p><p> participle: (primitive) | (The) |            (force) </p>array ([ERROR] = [WARNING] = [status] = 0 [Fields] = = Array (            [0] = = Username [1] = post [2] = summary) [Attrs] = = Array ( [UserID] = 1 [Username] = 7 [Addtime] = 2 [post] + 7) [Mat                    Ches] = Array ([0] = = Array ([id] = 20 [Weight] = 4500 [attrs] = = Array ([userid] =                            > 4 [Username] = Harry [Addtime] = 1375345494                    [post] = The force of the Primitive)) [1] = = Array ( [ID] [Weight] = 3451 [attrs] = = Array ([UserID] = 3 [Username] = John Doe                        [Addtime] = 1438460694 [post] = the primitive )) [2] = = Array ([id] = [weight                            ] = 2436 [Attrs] = Array ([userid] = 2                            [Username] = Zhangsi [Addtime] = 1408472694    [post] = Chinese)) [Total] = 3 [Total_found] + 3 [TIME] = 0 [words] = = Array ([flood] = = Array ([docs] =& Gt  4 [hits] + 4) [Araki] = Array ([docs]            = 3        [hits] + 3) [the] = = Array ([docs] = 2  [hits] = 2) [force] = = Array ([docs] = 1 [hits] = 1)))

4 Query Query.log



Sphinx + SCWS + Mysql + PHP Full Text Search

Related Article

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: info-contact@alibabacloud.com and provide relevant evidence. A staff member will contact you within 5 working days.

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.