Sphinxscws full-text retrieval and installation Configuration

Source: Internet
Author: User
We have previously set up a secondary sphinx. For details, refer to: sphinxmmsegmysql Chinese Word Segmentation coreseeksphinxmmsegmysql full-text search and installation. This environment is set up in centos6.5x86, php5.3.3, and mysql5.5.8, scws related package sphinx: sphinxsearc environment: centos 6.5x86, php 5.3.3, mysql 5.5.8


1. Download the sphinx and scws packages

Sphwheel: http://sphinxsearch.com/downloads/release/

Sph?php Extension: http://pecl.php.net/package/sphinx

Scws: http://www.xunsearch.com/scws/download.php

Scws dictionary: http://www.xunsearch.com/scws/down/scws-dict-chs-utf8.tar.bz2

Ii. Install sphinx, scws, and php extensions

1. Install sphinx

# tar zxvf sphinx-2.2.5-release.tar.gz  # cd sphinx-2.2.5-release  # ./configure --prefix=/usr/local/sphinx2 --with-mysql=/usr/local/mysql  # make && make install

2. Install the sphinx Client

View copy print?

# Cd api/libsphinxclient // sph‑2.2.5-release directory #./configure -- prefix =/usr/local/sphinx2/libsphinxclient # make & make install

Install the sphsf-php Extension

# Tar zxvf sphinx-1.3.1.tgz

# Cd sph0000- 1.3.1

# Phpize

#./Configure -- with-sphashes =/usr/local/sphinx2/libsphinxclient -- with-php-config =/usr/bin/php-config

# Make & make install

4. Install scws

View copy print?

# tar xvjf scws-1.2.2.tar.bz2  # mkdir /usr/local/scws  # cd scws-1.2.2  # ./configure --prefix=/usr/local/scws/  # make && make install

5. Install scws php Extension

# cd ./phpext/  # phpize  # ./configure --with-php-config=/usr/bin/php-config  # make && make install

3. Configure sphinx, scws, php, etc.

1. Create test tables and data

Mysql> desc users; + ---------- + ------------- + ------ + ----- + --------- + ---------------- + | Field | Type | Null | Key | Default | Extra | + ---------- + ------------- + ------ + ----- + --------- + ---------------- + | user_id | int (11) | NO | PRI | NULL | auto_increment | username | varchar (20) | NO | NULL | + ---------- + ------------- + ------ + ----- + --------- + ---------------- + 2 rows in set (0.00 sec) mysql> select * from users; + ------------ + | user_id | username | + ------------ + | 1311895262 | Zhang San | 1311895263 | tank Zhang 'er | 1311895264 | tank Zhang Yi | 1311895265 | tank Zhang | + ------------ + ------------ + 4 rows in set (0.00 sec)


Mysql> desc users; + ---------- + ------------- + ------ + ----- + --------- + ---------------- + | Field | Type | Null | Key | Default | Extra | + ---------- + ------------- + ------ + ----- + --------- + ---------------- + | user_id | int (11) | NO | PRI | NULL | auto_increment | username | varchar (20) | NO | NULL | + ---------- + ------------- + ------ + ----- + --------- + ---------------- + 2 rows in set (0.00 sec) mysql> select * from users; + ------------ + | user_id | username | + ------------ + | 1311895262 | Zhang San | 1311895263 | tank Zhang 'er | 1311895264 | tank Zhang Yi | 1311895265 | tank Zhang | + ------------ + ------------ + 4 rows in set (0.00 sec)

The two tables above are all real mysql tables.

2. Configure sphworker. conf with the following content

source myorder  {   type = mysql   sql_host = localhost   sql_user = root   sql_pass =   sql_db = test   sql_query_pre = SET NAMES utf8   sql_query_pre = SET SESSION query_cache_type=OFF   sql_query = \   SELECT a.id, a.user_id,b.username, UNIX_TIMESTAMP(a.create_time) AS create_time, a.product_name, a.summary \   FROM orders a left join users b on a.user_id = b.user_id   sql_attr_uint = user_id   sql_field_string = username   sql_field_string = product_name   sql_attr_timestamp = create_time     sql_ranged_throttle = 0   #sql_query_info = SELECT * FROM orders WHERE id=$id  }    index myorder  {   source = myorder   path = /usr/local/sphinx2/var/data/myorder   docinfo = extern   mlock = 0   morphology = none   min_word_len = 1   #charset_type = zh_cn.utf-8   html_strip = 1   charset_table = U+FF10..U+FF19->0..9, 0..9, U+FF41..U+FF5A->a..z, U+FF21..U+FF3A->a..z,A..Z->a..z, a..z, U+0149, U+017F, U+0138, U+00DF, U+00FF, U+00C0..U+00D6->U+00E0..U+00F6,U+00E0..U+00F6, U+00D8..U+00DE->U+00F8..U+00FE, U+00F8..U+00FE, U+0100->U+0101, U+0101,U+0102->U+0103, U+0103, U+0104->U+0105, U+0105, U+0106->U+0107, U+0107, U+0108->U+0109,U+0109, U+010A->U+010B, U+010B, U+010C->U+010D, U+010D, U+010E->U+010F, U+010F,U+0110->U+0111, U+0111, U+0112->U+0113, U+0113, U+0114->U+0115, U+0115, U+0116->U+0117,U+0117, U+0118->U+0119, U+0119, U+011A->U+011B, U+011B, U+011C->U+011D, U+011D,U+011E->U+011F, U+011F, U+0130->U+0131, U+0131, U+0132->U+0133, U+0133, U+0134->U+0135,U+0135, U+0136->U+0137, U+0137, U+0139->U+013A, U+013A, U+013B->U+013C, U+013C,U+013D->U+013E, U+013E, U+013F->U+0140, U+0140, U+0141->U+0142, U+0142, U+0143->U+0144,U+0144, U+0145->U+0146, U+0146, U+0147->U+0148, U+0148, U+014A->U+014B, U+014B,U+014C->U+014D, U+014D, U+014E->U+014F, U+014F, U+0150->U+0151, U+0151, U+0152->U+0153,U+0153, U+0154->U+0155, U+0155, U+0156->U+0157, U+0157, U+0158->U+0159, U+0159,U+015A->U+015B, U+015B, U+015C->U+015D, U+015D, U+015E->U+015F, U+015F, U+0160->U+0161,U+0161, U+0162->U+0163, U+0163, U+0164->U+0165, U+0165, U+0166->U+0167, U+0167,U+0168->U+0169, U+0169, U+016A->U+016B, U+016B, U+016C->U+016D, U+016D, U+016E->U+016F,U+016F, U+0170->U+0171, U+0171, U+0172->U+0173, U+0173, U+0174->U+0175, U+0175,U+0176->U+0177, U+0177, U+0178->U+00FF, U+00FF, U+0179->U+017A, U+017A, U+017B->U+017C,U+017C, U+017D->U+017E, U+017E, U+0410..U+042F->U+0430..U+044F, U+0430..U+044F,U+05D0..U+05EA, U+0531..U+0556->U+0561..U+0586, U+0561..U+0587, U+0621..U+063A, U+01B9,U+01BF, U+0640..U+064A, U+0660..U+0669, U+066E, U+066F, U+0671..U+06D3, U+06F0..U+06FF,U+0904..U+0939, U+0958..U+095F, U+0960..U+0963, U+0966..U+096F, U+097B..U+097F,U+0985..U+09B9, U+09CE, U+09DC..U+09E3, U+09E6..U+09EF, U+0A05..U+0A39, U+0A59..U+0A5E,U+0A66..U+0A6F, U+0A85..U+0AB9, U+0AE0..U+0AE3, U+0AE6..U+0AEF, U+0B05..U+0B39,U+0B5C..U+0B61, U+0B66..U+0B6F, U+0B71, U+0B85..U+0BB9, U+0BE6..U+0BF2, U+0C05..U+0C39,U+0C66..U+0C6F, U+0C85..U+0CB9, U+0CDE..U+0CE3, U+0CE6..U+0CEF, U+0D05..U+0D39, U+0D60,U+0D61, U+0D66..U+0D6F, U+0D85..U+0DC6, U+1900..U+1938, U+1946..U+194F, U+A800..U+A805,U+A807..U+A822, U+0386->U+03B1, U+03AC->U+03B1, U+0388->U+03B5, U+03AD->U+03B5,U+0389->U+03B7, U+03AE->U+03B7, U+038A->U+03B9, U+0390->U+03B9, U+03AA->U+03B9,U+03AF->U+03B9, U+03CA->U+03B9, U+038C->U+03BF, U+03CC->U+03BF, U+038E->U+03C5,U+03AB->U+03C5, U+03B0->U+03C5, U+03CB->U+03C5, U+03CD->U+03C5, U+038F->U+03C9,U+03CE->U+03C9, U+03C2->U+03C3, U+0391..U+03A1->U+03B1..U+03C1,U+03A3..U+03A9->U+03C3..U+03C9, U+03B1..U+03C1, U+03C3..U+03C9, U+0E01..U+0E2E,U+0E30..U+0E3A, U+0E40..U+0E45, U+0E47, U+0E50..U+0E59, U+A000..U+A48F, U+4E00..U+9FBF,U+3400..U+4DBF, U+20000..U+2A6DF, U+F900..U+FAFF, U+2F800..U+2FA1F, U+2E80..U+2EFF,U+2F00..U+2FDF, U+3100..U+312F, U+31A0..U+31BF, U+3040..U+309F, U+30A0..U+30FF,U+31F0..U+31FF, U+AC00..U+D7AF, U+1100..U+11FF, U+3130..U+318F, U+A000..U+A48F,U+A490..U+A4CF   ngram_len = 1   ngram_chars = U+4E00..U+9FBF, U+3400..U+4DBF, U+20000..U+2A6DF, U+F900..U+FAFF,U+2F800..U+2FA1F, U+2E80..U+2EFF, U+2F00..U+2FDF, U+3100..U+312F, U+31A0..U+31BF,U+3040..U+309F, U+30A0..U+30FF,U+31F0..U+31FF, U+AC00..U+D7AF, U+1100..U+11FF,U+3130..U+318F, U+A000..U+A48F, U+A490..U+A4CF  }

Note that the new sphinx does not support SQL _query_info and charset_type,

WARNING: key 'SQL _ query_info' was permanently removed from sphsf-configuration. Refer to documentation for details.
WARNING: key 'charset _ type' was permanently removed from sphsf-configuration. Refer to documentation for details.

3. Install the scws dictionary

# tar xvjf scws-dict-chs-utf8.tar.bz2 -C /usr/local/scws/etc/    # chown tank:tank /usr/local/scws/etc/dict.utf8.xdb

You must add permissions here. That is to say, to allow the php-fpm or php-cgi running users to have all the permissions of dict. utf8.xdb. If you do not do this, the php extension will report the following error when calling the dictionary:

Warning: SimpleCWS: add_dict (): Failed to add the dict file

How can I check the running users of php-fpm and php-cgi?

# Ps aux | grep php-fpm root 23487 0.0 0.1 284928 4652? Ss Nov05 php-fpm: master process (/etc/php-fpm.conf) tank 23488 0.0 1.3 336108 52328? S Nov05 02 php-fpm: pool www // here is tank, tank 23489 0.0 0.8 310484 34028? S Nov05 php-fpm: pool www tank 23490 0.0 0.7 306620 30156? S Nov05 php-fpm: pool www tank 23491 0.0 0.8 310096 33748? S Nov05 php-fpm: pool www tank 23492 0.0 1.2 331812 47712? S Nov05 php-fpm: pool www tank 24669 0.0 1.2 333520 48896? S Nov05 01 php-fpm: pool www tank 29747 0.0 0.7 305000? S php-fpm: pool www tank 29761 0.0 1.0 320536 39928? S php-fpm: pool www root 30705 0.0 0.0 103260 872 pts/5 S + grep php-fpm

4. Configure php. ini

# vim /etc/php.ini  [sphinx]  extension = sphinx.so  [scws]  extension = scws.so  scws.default.charset = utf-8  scws.default.fpath = /usr/local/scws/etc

4. Start sphtracing and php-fpm

1. Start sphinx

# /usr/local/sphinx2/bin/indexer --config /usr/local/sphinx2/etc/sphinx.conf --all  # /usr/local/sphinx2/bin/searchd --config /usr/local/sphinx2/etc/sphinx.conf

2. Restart php-fpm.

# /etc/init.d/php-fpm restart

The first two times, I have to install the sphplugin in mysql, but this time I didn't.

5. Test sphinx full-text search

1. Command Line Testing

[Root @ localhost phpext] # mysql-h 127.0.0.1-P 9306 Welcome to the MySQL monitor. commands end with; or \ g. your MySQL connection id is 1 Server version: 2.2.5-id64-release (r4825) Copyright (c) 2000,201 3, Oracle and/or its affiliates. all rights reserved. oracle is a registered trademark of Oracle Corporation and/or its affiliates. other names may be trademarks of their respective owners. type 'help; 'or' \ H' for help. type '\ C' to clear the current input statement. mysql> select * from myorder where match ('zhang '); + ------ + ------------ + ------------- + response + | id | user_id | username | create_time | product_name | + ------ + ------------ + ------------- + response + | 9 | 1311895262 | Zhang San | 1406823894 | tank is tank | 10 | 1311895263 | tank Zhangji | 1406823894 | tank is tank | 11 | 1311895264 | tank zhangyi | 1406823894 | tank is tank | 12 | 1311895265 | tank zhang | 1406823894 | tank is tank | + ------ + ------------ + ------------- + ---------------- + 4 rows in set (0.00 sec)

2. Use php Extension

 '. $ B _time .'

'; $ Key = "zhangsan"; $ index = "myorder "; // ================================================ = word segmentation $ so = scws_new (); $ so-> set_charset ('utf-8'); // default dictionary $ so-> add_dict (ini_get ('scws. default. fpath '). '/dict. utf8.xdb '); // custom dictionary // $ so-> add_dict ('. /dd.txt ', SCWS_XDICT_TXT); // default rule $ so-> set_rule (ini_get ('scws. default. fpath '). '/rules. utf8.ini '); // sets whether to remove some special punctuation marks $ so-> set_ignore (true) when the word splitting returns results; // sets whether to perform multiple segmentation when the word splitting returns results, if "Chinese" returns "China ", + MAN + Chinese. // Bitwise exclusive or 1 | 2 | 4 | 8 represents short words | binary | main word | all words // 1, 2, corresponds to the constant SCWS_MULTI_SHORT 1_scws_multi_zmain SCWS_MULTI_ZALL $ so-> set_multi (false); // sets whether to automatically aggregate idle text by two-word segmentation $ so-> set_duality (false ); // set the search term $ so-> send_text ($ key); $ words_array = $ so-> get_result (); $ words = ""; foreach ($ words_array as $ v) {$ words = $ words. '| ('. $ v ['word']. ')';} // Add the full word # $ words = '('. $ key. ')'. $ words; $ words = trim ($ words, '|'); $ so-> close (); echo'

Input: '. $ key .'

'; Echo'

Word Segmentation: '. $ words .'

'; // ================================================ = SEARCH $ SC = new SphinxClient (); $ SC-> SetServer ('2017. 0.0.1 ', 9312); # $ SC-> SetMatchMode (SPH_MATCH_ALL); $ SC-> SetMatchMode (SPH_MATCH_EXTENDED); $ SC-> SetArrayResult (TRUE ); $ res = $ SC-> Query ($ words, $ index); print_r ($ res); $ e_time = microtime (true); $ time = $ e_time-$ B _time; echo'

'. $ E_time .'

'; Echo'

'. $ Time .'

'; Exit;?>

The result is as follows:

1415214126.9106

Input: James

Word Segmentation: (Zhang San)

Array ([error] => [warning] => [status] => 0 [fields] => Array ([0] => username [1] => product_name [2] => summary) [attrs] => Array ([user_id] => 1 [username] => 7 [create_time] => 2 [product_name] => 7) [matches] => Array ([0] => Array ([id] => 9 [weight] => 2500 [attrs] => Array ([user_id] => 1311895262 [username] => Zhang San [create_time] => 1406823894 [product_name] => tank is tank ))) [total] => 1 [total_found] => 1 [time] => 0 [words] => Array ([sheets] => Array ([docs] => 4 [hits] => 4) [3] => Array ([docs] => 1 [hits] => 1 )))

1415214126.9516

0.041085958480835

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: info-contact@alibabacloud.com and provide relevant evidence. A staff member will contact you within 5 working days.

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.