Install and configure Mysql + sphinx + Chinese Word Segmentation in ubuntu

Dark visitor

1. Compile previous conditions

Check whether the following software has been installed. Some of them may not be necessary, but we recommend that you install them all.

Apt-get install autoconf automake autotools-dev cpp curl gawk gcc lftp libc6-dev linux-libc-dev make libpcre3-dev libpcrecpp0 g ++ libtool libncurses5-dev

Aptitude install libmysql + +-dev libmysqlclient15-dev checkinstall

Apt-get install python-dev

Ii. files required for Installation

List of required files

Mmseg-0.7.3.tar.gz Chinese Word Segmentation

Mysql-5.1.26-rc.tar.gz mysql-5.1.26 source code

Sphinx-0.9.8-rc2.tar.gz sphinx-0.9.8-rc2 source code

Fix-crash-in-excerpts.patch sphinx support word segmentation patch

Sphinx-0.98rc2.zhcn-support.patch sphinx support word segmentation patch

Files can be downloaded from the official website. I will provide these files in the attachment.

3. Start Installation

First, upload the above files to the root directory.

1, mmseg-0.7.3 Installation

Unzip the Installation File tar-zxvf mmseg-0.7.3.tar.gz first

Cd mmseg-0.7.3/

./Configure

Make

Make install

Cd ../

Mmseg has been installed.

Run the mmseg command to check whether the installation is successful.

2. Install the MySQL 5.1.26-rc, Sphinx, and SphinxSE storage engines

Decompress the mysql and sphinx source files first.

Tar zxvf mysql-5.1.26-rc.tar.gz

Tar zxvf sphinx-0.9.8-rc2.tar.gz

Then patch the sphenders. This patch is required to support Chinese characters.

Cd sphinx-0.9.8-rc2/

(Verify that the patch has been installed. If no patch is installed, use apt-get install patch)

Patch-p1 <../sphinx-0.98rc2.zhcn-support.patch

Patch-p1 <../fix-crash-in-excerpts.patch

Then copy the data in the mysqlse folder under sphinx to the mysql-5.1.26-rc/storage/sphinx (so that the SphinxSE storage engine can be compiled when mysql is compiled)

Cp-rf mysqlse ../mysql-5.1.26-rc/storage/sphse

Go to the/mysql-5.1.26-rc/storage/sphinx directory to check if the file has been copied successfully

Cd ../

Next let's compile and install the mysql-5.1.26-rc

Cd mysql-5.1.26-rc/

Sh BUILD/autorun. sh is required. Do not omit it.

Use the configure command to confirm Installation

./Configure -- prefix =/usr/local/mysql -- with-charset = utf8 -- with-extra-charsets = all -- with-plugins = sphsets

Check whether a prompt can be compiled using make. If yes.

Run

Make

This step takes a long time, generally in 10-20 minutes, mainly depends on the personal computer configuration

After the compilation is successful, run the installation command again (if an error occurs, you need to re-compile it. Generally, 80% of errors are not installed in the compilation environment (my compilation experience ))

Make install // This process takes less than 1 minute

Now that mysql has been installed, We will configure and start mysql.

Add mysql user groups and users

Groupadd mysql

Useradd-g mysql

Add mysql user permissions to the directory where mysql is located

Chown mysql: mysql/usr/local/mysql-R

Copy the my-medium.cnf to the installation root directory

Cp/usr/local/mysql/share/mysql/my-medium.cnf/usr/local/mysql/my. cnf

Add mysql user permissions to the my. cnf File

Chown mysql: mysql/usr/local/mysql/my. cnf

Then modify the configuration of my. cnf

Insert it on skip-external-locking (the configuration can be customized)

User = mysql

Pid-file =/usr/local/mysql. pid

Socket =/tmp/mysq. sock

Port = 3306

Basedir =/usr/local/mysql

Datadir =/usr/local/mysql/data

Tmpdir =/tmp

Language =/usr/local/mysql/share/mysql/English

Replace
-Comment out skin-dbd. Otherwise, an error is reported when the mysql Command is started.

To facilitate the use of mysql in ubuntu, we also need to install the mysql-client-5.0 (client)

Apt-get install mysql-client-5.0

Initialize Database

/Usr/local/mysql/bin/mysql_install_db -- defaults-file =/usr/local/mysql/my. cnf -- user = mysql

Execution result

Installing MySQL system tables...

080917 14:36:16 [Warning] Storage engine sphsf-has conflicting typecode. Assigning value 42.

OK

Filling help tables...

080917 14:36:16 [Warning] Storage engine sphsf-has conflicting typecode. Assigning value 42.

OK

....................

The preceding prompt indicates that the execution is successful.

At this time, you will find that the/user/local/mysql directory has a data folder, which stores the mysql data content.

We need to grant the user and write permissions to this directory.

Chown mysql: mysql-R/usr/local/mysql/data/

Chmod-R o + w/usr/local/mysql/data/

Copy mysql. server to the/etc/init. d/mysqld File

Cp/usr/local/mysql/share/mysql. server/etc/init. d/mysqld

And add execution right to mysqld

Chmod 700/etc/init. d/mysqld

Run

/Etc/init. d/mysqld start to start mysql, you can also use stop | restart and other commands

Then we enter the mysql command and enter

Mysql>

Input

Show engines;

Now we can see that the sphinxSE engine is included.

The myql part has been started.

Next let's install sphinx-0.9.8-rc2

Cd sphinx-0.9.8-rc2/

CPPFLAGS =-I/usr/include/python2.4

LDFLAGS =-lpython2.4

./Configure -- prefix =/usr/local/sphure -- with-mysql =/usr/local/mysql

Make

Make install

Cd ../

After installation is complete

Copy sphworker. conf. dist to the sphworker. conf file.

Cp/usr/local/sphinx/etc/sphinx. conf. dist/usr/local/sphinx/etc/sphinx. conf

Modify the configuration in sphworker. conf as follows: (the configuration can be customized)

SQL _host = localhost

SQL _user = root

SQL _pass =

SQL _db = test

SQL Port = 3306

Load Index

/Usr/local/sphinx/bin/indexer -- config/usr/local/sphinx/etc/sphinx. conf -- all

Note: you must first create a soft link for libmysqlclient. so.16

Ln-s/usr/local/mysql/lib/mysql/libmysqlclient. so.16/usr/lib/libmysqlclient. so.16

Then pass

/Usr/local/sphinx/bin/searchd -- config/usr/local/sphinx/etc/sphinx. conf enable index listening

For the convenience of the following tests, we will first import the example. SQL script that comes with sphexample. conf into mysql.

Mysql </usr/local/sphtasks/etc/example. SQL // creates a test database and creates the documents test table and data.

By now, the installation is almost complete, and we will talk about their usage.

4. Start Using

1. Use SphinxSE to call Sphinx in mysql

First, create an index dedicated table:

De> create table 'sphregion '(
'Id' int (11) not null,
'Weight' int (11) not null,
'Query' varchar (255) not null,
'Catalogid' int not null,
'Edituserid' int not null,
'Others' int null,
'Addtime' int not null, KEY
'Query' ('query ')
) ENGINE = sphinx default charset = utf8 CONNECTION = sphinx: // localhost: 3312/test1de>

Test1: Index name, which can be found in sphexample. conf.

After creating an index dedicated table, we can use it in mysql. For example, enter

SELECT doc. * FROM documents doc JOIN sph0000on (doc. id = sph0000. id)

WHERE query = doc; mode = any

After running, the record line containing the doc string will be displayed in the result record

For more information about the query syntax and sphinx configurations, see:

Html> http://www.sphinxsearch.com/doc.html

Here is a very detailed introduction. Here we will only make a brief introduction, hoping to make some valuable achievements.

2. Application of Chinese Word Segmentation

Install apache2 and phpmyadmin before the application, so that you can enter Chinese characters for testing.

Apt-get install apache2

Apt-get install phpmyadmin

Access: