Gearman installation and use in Linux, distributed Message Queuing (centos-6.5:gearmand-1.1.12)

1 Gearman Introduction

1.1 Overview 

Gearman is a system that is used to delegate work to other machines, distributed calls that are more suitable for a particular job, concurrent workloads that do some work across multiple calls, or functions that are used to invoke other languages.

1.2 Composition

 Gearman is a distribution task of the program architecture, consisting of three parts:

1) Gearman client: Provides Gearman client API for application invocation. The API can use the C,php,perl,mysql UDF to wait for a language, which is the initiator of the request.
2) Gearman Job Server: Distributes the client's request to the dispatcher of each Gearman worker, equivalent to the central controller, but it does not process the specific business logic.
3) Gearman Worker: Provides the Gearman worker API to the application call, which is responsible for the client's request and returns the processing results to the client.

1.3 Applications

The core of MogileFS's Distributed file system is implemented with Gearman.

There are many applications for this software, such as video processing of video sites, distributed log processing, e-mail processing, file synchronization, image processing, and so on, as long as it can be released, not affect the experience and response of the scene, the need to parallel a lot of computation and processing procedures are possible. Yahoo uses Gearman on 60 or more servers to process 6 million jobs a day. News aggregator Digg has built a gearman network of the same size and can handle 400,000 jobs a day.

Gearman can not only be used as a task distribution, but also as a load balancer for applications. You can put the worker on a different heap of servers, or you can start on multiple cores on the same CPU. For example, the application of the video converter, do not want the Web server to handle video format conversion, at this time, you can do task distribution on this heap server, loading worker processing video format, external Web server will not be affected by the video conversion process. And the extension is convenient, add a server to the task Dispatch center, registered as a worker, then the job server will be the request arrives, the request is sent to the idle worker. You can also run multiple job servers to make up the HA architecture, and if one job server is dropped, the client and worker are automatically migrated to another job server.

1.4 Working principle Diagram

2. Operating Procedures

The processing of a Gearman request involves three roles: Client-to-Job Worker.

Client: The initiator of the request, which can be a c,php,perl,mysql UDF, and so on.
Job: The dispatcher of the request is responsible for coordinating the forwarding of requests from the Client to the appropriate work.
Worker: The handler for the request, which can be c,php,perl, and so on.
Because Client,worker does not restrict the use of the same language, it facilitates integration between multiple languages and multiple systems.
Even by adding more workers, we can easily implement the distributed load-balancing architecture of the application.

3. Gearman Download

 1) Official website

http://gearman.org/


2) Download the website
Https://launchpad.net/gearmand

3) Official website using the guide
http://gearman.org/getting-started/

4) All software used in this installation (installation environment is CentOS-6.5)
http://download.csdn.net/detail/clevercode/8698699

4 Gearman Installation
4.1 Installing the Linux Prerequisites Common Library
Installation of the necessary common support libraries in Linux: http://blog.csdn.net/clevercode/article/details/45438401


4.2 Installing Gearmand Dependent libraries # yum install-y boost-devel gperf libevent-devel libuuid-devel

4.3 Install Gearmand service

 1) Unzip

# Cd/usr/local/src/gearman
# tar Xzf gearmand-1.1.12.tar.gz

2) configuration
# CD gearmand-1.1.12
#./configure

3) Compiling
# make

4) Installation
# make Install

5) Installation success Diagram, enter

# Gearman

5 Installing PHP extensions

 1) Installing Phpize

# yum Install-y php-devel


2) Unzip
# Cd/usr/local/src/gearman
# tar Xzf gearman-1.1.2.tgz

3) configuration
# CD gearman-1.1.2
# phpize
#./configure

4) Compiling
# make

5) Installation
# make Install

6) Successful Installation

The "Installing Shared extensions:/usr/lib64/php/modules/" appears to indicate that the installation was successful and/usr/lib64/php/modules/is the directory of the gearman.so extension.

7) configuration (join extension)
# Vi/usr/local/php5/etc/php.ini
extension= "Gearman.so"

8) Check whether the configuration is successful
# VI test.php
<?php
Print gearman_version (). "\ n";
?>

After executing PHP test.php, a 1.1.12 appears indicating successful installation
# php test.php
1.1.12


6 Gearman Start stop

1) Create log/data0/logs/gearmand.log
# Touch/data0/logs/gearmand.log

2) Start
#/usr/local/sbin/gearmand-d-u root-l 192.168.142.130--log-file=/data0/logs/gearmand.log

3) Detailed parameters
Number of monitoring connections for-b,--backlog= Reserve
-D,--daemon background run
-F, number of--file-descriptors= file descriptors
-H,--help help
-j,--job-retries= the number of times the OB server has run before the unavailable job is removed, preventing the other available workers from crashing as a result of continuous operation. No Limit by default
-L,-log-file= log file location (the simplest log is the default record)
-L,--listen= listening IP, all accepted by default
-P,--port= specifies the listening port
-P,--pid-file= specifies the process ID write location
-R,--protocol= load protocol module
-Q,--queue-type= specifies the persistence queue
-T, the number of I/9 threads used by--threads=. Default is 0
-u, switch to the specified user after the--user= is started
-V,--verbose increase level of detail
-V,--version display version information

4) Check whether the operation
# PS Axu | grep Gearmand

5) Viewing the listening port
# NETSTAT-ANP | grep 4730

6) Stop and kill the process directly.



7 Gearman Use

7.1 Create worker Create worker.php to create a worker side to send mail to. The code is as follows

<?php

$worker= new GearmanWorker();
$worker->addServer('192.168.142.130', '4730');
$worker->addFunction("sendMail", "my_sendmail_function");
While ($worker->work());

Function my_sendmail_function($job){
    
     // Receive data
     $tmp = $job->workload();
     $receiveArr = unserialize($tmp);
    
     $from = $receiveArr['from'];
     $to = $receiveArr['to'];
     $subject = $receiveArr['subject'];
     $content = $receiveArr['content'];
    
     //send email
     //....
    
     Return $subject.' sendmail OK';
}

?>

7.2 Starting the worker side

If the amount of data processed is large, you can execute the following script multiple times, starting multiple worker ends.

# nohup PHP worker.php > Tmp.txt &


7.3 Create a client (blocking mode, you need to wait for the return result to end) to create a client.php. The Do () method is blocking mode and must wait for the worker side to return results before the program can stop. (return: Hello Gearman sendmail OK)

<?php

$client= new GearmanClient();
$client->addServer('192.168.142.130', '4730');

$job = array();
$job['from'] = 'CleverCode';
$job['to'] = 'Gearman';
$job['subject'] = 'hello Gearman';
$job['content'] = 'hello Gearman:this is from GearmanClient';
$job = serialize($job);

/ / Wait until the worker returns the result, it will end.
$ret = $client->do("sendMail", $job);

Echo $ret."\r\n";

?>

7.4 Create a client (non-blocking, without waiting for results) to create a client2.php. Dobackground () The program ends without waiting for the worker side to return the results.

<?php

$client= new GearmanClient();
$client->addServer('192.168.142.130', '4730');

$job = array();
$job['from'] = 'CleverCode';
$job['to'] = 'Gearman';
$job['subject'] = 'hello Gearman';
$job['content'] = 'hello Gearman:this is from GearmanClient';
$job = serialize($job);

/ / Do not wait for the return result, it will end
$ret = $client->doBackground("sendMail", $job);
Echo $ret."\r\n";

?> 

8 Gearman Management 

Enter the following command to view the 4730 port condition.

# (echo "status"; Sleep 2) | Telnet 192.168.142.130 4730

1) field Description: "Worker is available" for tasks in the "Running tasks" queue known as registered tasks.
2) sendMail 0 0 1, the registered task named sendmail,0 is running normally, the queue is empty, there is an available Worker.

Copyright Notice:

1) original works, from "Clevercode's blog" , please be sure to mention the following original address when reproduced , otherwise hold the copyright legal responsibility.

2) Original address : http://blog.csdn.net/clevercode/article/details/45718735 ( reprint must indicate this address ).

3) Blog column address (Linux common software Installation and configuration): http://blog.csdn.net/column/details/linuxsoftwareinstall.html (continue to increase, concern please bookmark).

4) welcome everyone to pay attention to my blog more wonderful content: Http://blog.csdn.net/CleverCode.

Gearman installation and use in Linux, distributed Message Queuing (centos-6.5:gearmand-1.1.12)