Gearman implementing Redis Cache MySQL

Gearman Introduction:

Gearman is a distributed task-distribution framework. The design is simple and has been widely supported. A typical Gearman application consists of the following sections:

1, Gearman Job Server:gearman Core program, in the form of daemons running in the background

2, Gearman Client: Can be understood as the task of the recipient, such as I want to perform a task in the background to send mail, you can call a Gearman Client in the program and incoming message information, and then you can immediately display the results of the execution to the user, and the task itself will slowly run in the background.

3, Gearman worker: The real performer of the task, generally need to write their own specific logic and through the daemon mode of operation, 4, Gearman Worker receives Gearman client to pass the task content, will be processed sequentially.

Design ideas:

First, the MySQL UDF (implemented by the combination of Lib_mysqludf_json and GEARMAN-MYSQL-UDF) triggers the trigger to pass data into the Gearman when the data in MySQL is changed. At this time, MySQL is equivalent to Gearman's clinet. Then run your own PHP program as a worker, the data in the Gearman to Redis, when the Redis equivalent is the Gearman consumer.

Implementation process:

1, configure the Yum source, install Gearmand

http://down.51cto.com/data/2274661 # #YUM包文件配置, and Gearmand need the package

1. Yum install-y php-pecl-gearman libgearman libgearman-devel gearmand NC

2. Starting Service/etc/init.d/gearmand Start

3, check whether the status is OK NETSTAT-ALNUTP |grep Gearman

# # #gearman默认使用的端口为4370

2, installation database, Php-mysql plug-in

1, database installation (slightly)

2, plug-in installation Yum-yinstall Php-mysql #安装这个插件主要是用PHP来数据收集写入操作

3. Start MySQL

3. Install Lib_mysqludf_json on the data server

1, Unzip Master.zip

2. CD Lib_mysqludf_json-master

3, Rm-rf lib_mysqludf_json.so

4, GCC $ (mysql_config--cflags)-shared-fpic-o lib_mysqludf_json.solib_mysqludf_json.c

# #把重新编译的lib_mysqludf_json. So copy to the/usr/local/mysql/lib/plugin directory

4. Install GEARMAN-MYSQL-UDF on the data server

1. Unzip the package tar XF gearman-mysql-udf-0.6.tar.gz-c.

2. Cd gearman-mysql-udf-0.6

3, compile./configure--with-mysql=/usr/local/mysql/bin/mysql_config--libdir=/usr/local/mysql/lib/plugin/

#注意这是自己定义的数据安装包的目录, if installed with the Yum package

./configure--with-mysql=/usr/bin/mysql_config--libdir=/usr/lib64/plugin/

Compile error: configure:error:At least version 0.33 of Libgearman is required for GEARMAN-MYSQL-UDF

Installation: Yum-y installlibgearman-devel #由于原来包版本低, re-install it.

5, install the make&& make install # #插件安装到MySQL的插件目录下.

6, enter the database, create the corresponding Function,trigger and set up gearmanserver information

1, createfunction Json_object returns string Soname ' Lib_mysqludf_json.so '

2. Create Functiongman_do_background returns string Soname ' Libgearman_mysql_udf.so '

3, Createfunction Gman_servers_set returns string Soname ' libgearman_mysql_udf.so ';

7, installation redis# recommended to use the 3.0.7 version and install PHP connection Redis driver, test environment with Yum installation Redis

1. Yum Install Php-pecl-redis Redis–y

2. Start Redis service Servie Redis start

3, login Redis Simple Authentication

[email protected] gearman-mysql-udf-0.6] #redis-cli

Redis 127.0.0.1:6379> Keys *

1) "counter:__rand_int__"

2) "key:__rand_int__"

3) "Save"

4) "MyList"

8, log in the database, according to the business of a library under a table to add a trigger, and set Gearman server's letter

1,/usr/local/mysql/bin/mysql-s/tmp/mysql3306.sock-p

2. Create DATABASE WY

3. CREATE table Test (Idint primary Key,name char (20))

4, set Gearman server information # #非常重要, if not executed will result in data cannot be collected into Redis

MariaDB [wy]> selectgman_servers_set (' 127.0.0.1:4730 ');

+------------------------------------+

| Gman_servers_set (' 127.0.0.1:4730 ') |

+------------------------------------+

| 127.0.0.1:4730 |

+------------------------------------+

9. Add Trigger

DELIMITER $$

CREATE TRIGGER Datatoredis after UPDATE on test for each ROW BEGIN

[Email protected]=gman_do_background (' Synctoredis ', Json_object (new.id as ' id ', new.name as ' name '));

end$$

DELIMITER;

10, write a worker program, responsible for the data in the Gearman into Redis:

File name: redis_workder.php

1. Write:

2. Start phpredis_worker.php

Start error: Phpwarning:gearmanworker::work (): Send_packet (Gearman_errno) Failed to send server-options packet libgearman/connection.cc:485 in/home/redis_worker.php on line 8

Workaround:/usr/sbin/gearmand-d-L 127.0.0.1-p 4730

Then at startup: Nohup php redis_worker.php & keep running in the background

start Gearmand error/var/log/mearmond.log bug not resolved

ERROR 2016-12-28 07:46:38.000000 [main] socket (Address family not supported by protocol)->libgearman-server/ge armand.cc:468

ERROR 2016-12-28 07:46:38.000000 [main] Gearmand_sockfd_close () called with aninvalid socket--Libgearman-ser ver/io.cc:933

Summarize:

Testing by writing data in the MySQL database, and then observing whether Redis has data, if any, the efficiency is very low, in the 100W data, database write performance is greatly reduced.

This article is from the "DBSpace" blog, so be sure to keep this source http://dbspace.blog.51cto.com/6873717/1887632

Gearman implementing Redis Cache MySQL