Sphinx SCWS Full-Text Search installation configuration steps

Source: Internet
Author: User
Tags bz2 documentation fpm ini

The setting of the environment: CentOS 6.5 x86,php 5.3.3,mysql 5.5.8, three times to build each different
one, download SPHINX,SCWS related packages
Sphinx Download Address: http ://sphinxsearch.com/downloads/release/
Sphinx php Extended Download Address: Http://pecl.php.net/package/sphinx
Scws Download Address: http ://www.xunsearch.com/scws/download.php
Scws Word library download address: http://www.xunsearch.com/scws/down/ SCWS-DICT-CHS-UTF8.TAR.BZ2
Two, installing SPHINX,SCWS, and PHP extensions
1, installing Sphinx

The code is as follows Copy Code
# tar ZXVF sphinx-2.2.5-release.tar.gz
# CD Sphinx-2.2.5-release
#./configure--prefix=/usr/local/sphinx2--with-mysql=/usr/local/mysql
# Make && make install

2, install Sphinx Client

The code is as follows Copy Code
# CD Api/libsphinxclient//sphinx-2.2.5-release directory
#./configure--prefix=/usr/local/sphinx2/libsphinxclient
# Make && make install

3, install Sphinx PHP extension

The code is as follows Copy Code
# tar ZXVF sphinx-1.3.1.tgz
# CD sphinx-1.3.1
# phpize
#./configure--with-sphinx=/usr/local/sphinx2/libsphinxclient--with-php-config=/usr/bin/php-config
# Make && make install

4, install SCWS

The code is as follows Copy Code
# tar XVJF scws-1.2.2.tar.bz2
# MKDIR/USR/LOCAL/SCWS
# CD scws-1.2.2
#./configure--prefix=/usr/local/scws/
# Make && make install

5, install scws PHP extension

The code is as follows Copy Code
# CD./phpext/
# phpize
#./configure--with-php-config=/usr/bin/php-config
# Make && make install

Third, configure sphinx,scws,php, etc.
1, create test tables and data

The code is as follows Copy Code
mysql> desc users;


+----------+-------------+------+-----+---------+----------------+


| Field | Type | Null | Key | Default | Extra |


+----------+-------------+------+-----+---------+----------------+


| user_id | Int (11) | NO | PRI | NULL | auto_increment |


| Username | varchar (20) | NO | | NULL | |


+----------+-------------+------+-----+---------+----------------+


2 rows in Set (0.00 sec)





Mysql> select * from users;


+------------+------------+


| user_id | Username |


+------------+------------+


| 1311895262 | John |


| 1311895263 | Tank Zhang Yi |


| 1311895264 | Tank Zhang |


| 1311895265 | Tank Zhang |


+------------+------------+


4 rows in Set (0.00 sec)


mysql> desc orders;


+--------------+-------------+------+-----+---------+----------------+


| Field | Type | Null | Key | Default | Extra |


+--------------+-------------+------+-----+---------+----------------+


| ID | Int (11) | NO | PRI | NULL | auto_increment |


| user_id | Int (11) | NO | | NULL | |


| Create_time | datetime | NO | | NULL | |


| Product_Name | varchar (20) | NO | | NULL | |


| Summary | Text | NO | | NULL | |


+--------------+-------------+------+-----+---------+----------------+


5 rows in Set (0.00 sec)





Mysql> select * from Orders;


+----+------------+---------------------+----------------+--------------+


| ID | user_id | Create_time | Product_Name | Summary |


+----+------------+---------------------+----------------+--------------+


| 9 | 1311895262 | 2014-08-01 00:24:54 | Tank is Tank | Technical Director |


| 10 | 1311895263 | 2014-08-01 00:24:54 | Tank is Tank | Technical Manager |


| 11 | 1311895264 | 2014-08-01 00:24:54 | Tank is Tank | DNB Manager |


| 12 | 1311895265 | 2014-08-01 00:24:54 | Tank is Tank | Director of operation and Maintenance |


+----+------------+---------------------+----------------+--------------+


4 rows in Set (0.00 sec)


Above two tables, are real MySQL tables
2, configure sphinx.conf, plus the following content

The code is as follows Copy Code
SOURCE Myorder


{


Type = MySQL


Sql_host = localhost


Sql_user = root


Sql_pass =


sql_db = Test


Sql_query_pre = SET NAMES UTF8


Sql_query_pre = SET Session Query_cache_type=off


Sql_query =


SELECT a.id, A.user_id,b.username, Unix_timestamp (a.create_time) as Create_time, A.product_name, a.summary


From the orders a LEFT join the users b on a.user_id = b.user_id


Sql_attr_uint = user_id


Sql_field_string = Username


sql_field_string = Product_Name


Sql_attr_timestamp = Create_time





Sql_ranged_throttle = 0


#sql_query_info = SELECT * FROM orders WHERE id= $id


}





Index Myorder


{


Source = Myorder


Path =/usr/local/sphinx2/var/data/myorder


DocInfo = extern


Mlock = 0


Morphology = None


Min_word_len = 1


#charset_type = Zh_cn.utf-8


Html_strip = 1


 charset_table = U+ff10. u+ff19->0..9, 0..9, u+ff41. U+ff5a->a.. Z, U+FF21. U+ff3a->a.. Z,a.. Z->a.. Z, A.. Z, u+0149, u+017f, u+0138, U+00DF, U+00FF, u+00c0. U+00d6->u+00e0.. U+00f6,u+00e0.. U+00f6, U+00d8. U+00de->u+00f8.. U+00fe, U+00f8. U+00fe, u+0100->u+0101, u+0101,u+0102->u+0103, u+0103, u+0104->u+0105, u+0105, U+0106->U+0107, U+0107, U+ 0108->u+0109,u+0109, u+010a->u+010b, u+010b, u+010c->u+010d, u+010d, u+010e->u+010f, U+010F,U+0110->U +0111, u+0111, u+0112->u+0113, u+0113, u+0114->u+0115, u+0115, u+0116->u+0117,u+0117, U+0118->U+0119, U+ 0119, u+011a->u+011b, u+011b, u+011c->u+011d, u+011d,u+011e->u+011f, u+011f, u+0130->u+0131, U+0131, U+ 0132->u+0133, u+0133, u+0134->u+0135,u+0135, u+0136->u+0137, u+0137, u+0139->u+013a, U+013A, U+013B->U +013c, u+013c,u+013d->u+013e, u+013e, u+013f->u+0140, u+0140, u+0141->u+0142, U+0142, U+0143->U+0144,U+ 0144, u+0145->u+0146, u+0146, u+0147->u+0148, u+0148, u+014a->u+014b, u+014b,u+014c->u+014d, u+014d, u+014e->u+014f, u+014f, U+0150->U+0151, U+0151 , u+0152->u+0153,u+0153, u+0154->u+0155, u+0155, u+0156->u+0157, u+0157, u+0158->u+0159, U+0159,U+015A- >u+015b, u+015b, u+015c->u+015d, u+015d, u+015e->u+015f, u+015f, u+0160->u+0161,u+0161, U+0162->U+0163 , u+0163, u+0164->u+0165, u+0165, u+0166->u+0167, u+0167,u+0168->u+0169, u+0169, U+016A->U+016B, U+016B, U+ 016C->U+016D, u+016d, u+016e->u+016f,u+016f, u+0170->u+0171, u+0171, u+0172->u+0173, U+0173, U+0174->U +0175, u+0175,u+0176->u+0177, u+0177, u+0178->u+00ff, u+00ff, u+0179->u+017a, u+017a, U+017B->U+017C,U+ 017C, u+017d->u+017e, u+017e, u+0410..u+042f->u+0430..u+044f, u+0430..u+044f,u+05d0. U+05ea, u+0531..u+0556->u+0561..u+0586, u+0561..u+0587, u+0621..u+063a, U+01B9,U+01BF, U+0640..U+064A, U+0660..U+ 0669, u+066e, u+066f, U+0671..u+06d3, u+06f0. u+06ff,u+0904..u+0939, u+0958..u+095f, u+0960.. u+0963, u+0966..u+096f, u+097b. U+097F,U+0985..U+09B9, U+09ce, U+09DC. U+09e3, U+09e6. U+09ef, U+0a05. U+0a39, U+0a59. U+0a5e,u+0a66.. u+0a6f, U+0a85. U+0AB9, U+0ae0. U+0ae3, U+0ae6. U+0AEF, U+0b05. u+0b39,u+0b5c.. U+0b61, u+0b66. u+0b6f, U+0b71, u+0b85. U+0BB9, U+0be6. U+0BF2, U+0C05. U+0c39,u+0c66.. u+0c6f, u+0c85. U+0CB9, U+0CDE. U+0ce3, U+0ce6. U+0CEF, u+0d05. U+0d39, U+0d60,u+0d61, U+0d66. u+0d6f, u+0d85. U+0DC6, u+1900..u+1938, u+1946..u+194f, u+a800. u+a805,u+a807.. u+a822, U+0386->U+03B1, u+03ac->u+03b1, U+0388->u+03b5, U+03ad->u+03b5,u+0389->u+03b7, U+03AE->U+ 03b7, U+038A->U+03B9, U+0390->U+03B9, U+03AA->U+03B9,U+03AF->U+03B9, U+03CA->U+03B9, U+038C->U+ 03BF, U+03CC->U+03BF, U+038e->u+03c5,u+03ab->u+03c5, U+03b0->u+03c5, U+03CB->U+03C5, U+03CD->U+ 03C5, U+038F->U+03C9,U+03CE->U+03C9, U+03C2->U+03C3, u+0391..u+03a1->u+03b1. U+03c1,u+03a3.. U+03a9->u+03c3.. U+03C9, u+03b1. U+03C1, U+03C3. U+03C9, U+0e01. U+0e2e,u+0e30.. U+0E3A, U+0e40. U+0e45, U+0e47, U+0e50. U+0e59, u+a000. u+a48f, u+4e00. U+9FBF,U+3400..U+4DBF, U+20000..U+2A6DF, u+f900. U+faff, u+2f800. U+2FA1F, U+2e80. U+2eff,u+2f00.. U+2FDF, u+3100..u+312f, u+31a0. U+31BF, u+3040..u+309f, u+30a0. U+30ff,u+31f0.. U+31FF, u+ac00. U+D7AF, u+1100..u+11ff, u+3130..u+318f, u+a000. u+a48f,u+a490.. u+a4cf 


Ngram_len = 1


Ngram_chars = u+4e00. U+9FBF, U+3400..U+4DBF, U+20000..U+2A6DF, u+f900. u+faff,u+2f800.. U+2FA1F, U+2e80. U+2eff, u+2f00. U+2FDF, u+3100..u+312f, u+31a0. u+31bf,u+3040..u+309f, u+30a0. U+30ff,u+31f0.. U+31FF, u+ac00. U+D7AF, u+1100..u+11ff,u+3130..u+318f, u+a000. u+a48f, u+a490. U+a4cf


}


Note that the new Sphinx, which does not support sql_query_info,charset_type settings,

The code is as follows Copy Code
Warning:key ' Sql_query_info ' is permanently removed from Sphinx configuration. Refer to documentation for details.
Warning:key ' Charset_type ' is permanently removed from Sphinx configuration. Refer to documentation for details.

3, install SCWS Word Library

The code is as follows Copy Code
# tar XVJF scws-dict-chs-utf8.tar.bz2-c/usr/local/scws/etc/

# chown Tank:tank/usr/local/scws/etc/dict.utf8.xdb

Be sure to add permissions here, that is, let PHP-FPM or php-cgi run the user, have all the Dict.utf8.xdb permissions. If you do not, the PHP extension call Thesaurus will report the following error:
warning:simplecws::add_dict (): Failed to add the Dict file
How do I view php-fpm,php-cgi's running users?

The code is as follows Copy Code
# PS aux |grep php-fpm
Root 23487 0.0 0.1 284928 4652? Ss Nov05 0:00 php-fpm:master process (/etc/php-fpm.conf)
Tank 23488 0.0 1.3 336108 52328? S Nov05 0:02 Php-fpm:pool www//Here's tank.
Tank 23489 0.0 0.8 310484 34028? S Nov05 0:02 Php-fpm:pool www
Tank 23490 0.0 0.7 306620 30156? S Nov05 0:02 Php-fpm:pool www
Tank 23491 0.0 0.8 310096 33748? S Nov05 0:02 Php-fpm:pool www
Tank 23492 0.0 1.2 331812 47712? S Nov05 0:02 Php-fpm:pool www
Tank 24669 0.0 1.2 333520 48896? S Nov05 0:01 Php-fpm:pool www
Tank 29747 0.0 0.7 305000 27340? S 03:27 0:00 Php-fpm:pool www
Tank 29761 0.0 1.0 320536 39928? S 03:27 0:00 Php-fpm:pool www
Root 30705 0.0 0.0 103260 872 pts/5 s+ 04:11 0:00 grep php-fpm

4, configure PHP.ini

The code is as follows Copy Code
# Vim/etc/php.ini
[Sphinx]
Extension = sphinx.so

[SCWS]
Extension = scws.so
Scws.default.charset = Utf-8
Scws.default.fpath =/usr/local/scws/etc

Four, start sphinx,php-fpm

The code is as follows Copy Code
1, start Sphinx
#/usr/local/sphinx2/bin/indexer--config/usr/local/sphinx2/etc/sphinx.conf--all
#/usr/local/sphinx2/bin/searchd--config/usr/local/sphinx2/etc/sphinx.conf
2, restart PHP-FPM
#/etc/init.d/php-fpm Restart
The first two times, I install Sphinx, must install the Sphinx storage plugin in MySQL, but this time did not, look below figure
MySQL does not install Sphinx plugin
MySQL does not install Sphinx plugin

Five, test Sphinx Full-text Search
1, the command line of the test

The code is as follows Copy Code
[Root@localhost phpext]# mysql-h 127.0.0.1-p 9306


Welcome to the MySQL Monitor. Commands End With; or G.


Your MySQL Connection ID is 1


Server Version:2.2.5-id64-release (r4825)





Copyright (c), 2013, Oracle and/or its affiliates. All rights reserved.





Oracle is a registered trademark the Oracle Corporation and/or its


Affiliates. Names may trademarks of their respective


Owners.





Type ' help, ' or ' h ' for help. Type ' C ' to clear the current input statement.





Mysql> SELECT * from Myorder where match (' Zhang ');


+------+------------+------------+-------------+----------------+


| ID | user_id | Username | Create_time | Product_Name |


+------+------------+------------+-------------+----------------+


| 9 | 1311895262 | John | 1406823894 | Tank is Tank |


| 10 | 1311895263 | Tank Zhang Yi | 1406823894 | Tank is Tank |


| 11 | 1311895264 | Tank Zhang | 1406823894 | Tank is Tank |


| 12 | 1311895265 | Tank Zhang | 1406823894 | Tank is Tank |


+------+------------+------------+-------------+----------------+


4 rows in Set (0.00 sec)


2, using PHP to expand

The code is as follows Copy Code
<?php


Header ("content-type:text/html; Charset=utf-8 ");


$b _time = Microtime (true);


Echo ' <p> ' $b _time. ' </p> ';


$key = "John";


$index = "Myorder";


======================================== participle





$so = Scws_new ();


$so->set_charset (' utf-8 ');


Default Word Library


$so->add_dict (Ini_get (' Scws.default.fpath '). '/dict.utf8.xdb ');


Custom Word Library


$so->add_dict ('./dd.txt ', scws_xdict_txt);


Default rule


$so->set_rule (Ini_get (' Scws.default.fpath '). '/rules.utf8.ini ');





Set whether to remove some special punctuation marks when the word is returned to the result


$so->set_ignore (TRUE);





Set the word back to the results of the duplex division, such as "Chinese" return "Chinese + people + Chinese" three words.


Bitwise XOR 1 | 2 | 4 | 8 Representations: Short words | binary | Main Word | All Word


1,2,4,8 corresponding constant Scws_multi_short scws_multi_duality scws_multi_zmain Scws_multi_zall


$so->set_multi (FALSE);





Set whether to automatically aggregate the idle text in two Word segmentation method


$so->set_duality (FALSE);





Set search terms


$so->send_text ($key);


$words _array = $so->get_result ();


$words = "";


foreach ($words _array as $v)


{


$words = $words. ' | ('. $v [' word ']. ') ';


}





Add Whole Word


# $words = ' ('. $key. ') '. $words;


$words = Trim ($words, ' | ');


$so->close ();


Echo ' <p> input: '. $key. ' </p> ';


Echo ' <p> participle: '. $words. ' </p> ';


======================================== Search


$SC = new Sphinxclient ();


$SC->setserver (' 127.0.0.1 ', 9312);


# $SC->setmatchmode (Sph_match_all);


$SC->setmatchmode (sph_match_extended);


$SC->setarrayresult (TRUE);


$res = $sc->query ($words, $index);


Print_r ($res);


$e _time = Microtime (true);


$time = $e _time-$b _time;


Echo ' <p> ' $e _time. ' </p> ';





Echo ' <p> '. $time. ' </p> ';


Exit


?>


The results are as follows:


<p>1415214126.9106</p><p> input: John </p><p> participle: (John) </p>array


(


[ERROR] =>


[WARNING] =>


[Status] => 0


[Fields] => Array


(


[0] => username


[1] => Product_Name


[2] => Summary


)





[Attrs] => Array


(


[USER_ID] => 1


[Username] => 7


[Create_time] => 2


[Product_Name] => 7


)





[Matches] => Array


(


[0] => Array


(


[ID] => 9


[Weight] => 2500


[Attrs] => Array


(


[user_id] => 1311895262


[Username] => John


[Create_time] => 1406823894


[Product_Name] => tank is tank


)





)





)





[Total] => 1


[Total_found] => 1


[Time] => 0


[Words] => Array


(


[Zhang] => Array


(


[Docs] => 4


[Hits] => 4


)





[Three] => Array


(


[Docs] => 1


[Hits] => 1


)





)





)


<p>1415214126.9516</p><p>0.041085958480835</p>

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: info-contact@alibabacloud.com and provide relevant evidence. A staff member will contact you within 5 working days.

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.