Distributed task distribution framework Gearman tutorial and PHP implementation example

Distributed task distribution framework Gearman tutorial and PHP implementation example from: http://blog.csdn.net//clh604/article/details/19706827

1. introduction and use cases of Gearman

Gearman is a program framework for task distribution. it can be used in various scenarios. compared with Hadoop, Gearman prefers task distribution. Its task distribution is very simple, which can be simply completed by using scripts. Gearman was originally used for the image resize function of LiveJournal. because the image resize consumes a lot of computing resources, it needs to be scheduled to run on multiple backend servers. after the task is completed, it returns to the front end and then presents it to the interface.

Generally, multi-language and multi-system integration is a big problem. Generally, most people use WebService to solve such integration problems. However, no matter what type of WebService, such as RPC, or REST style. In contrast, Gearman can also achieve similar functions and is easier to use.


The processing of a Gearman request involves three roles: Client-> Job-> Worker.


Client: The request initiator, which can be C, PHP, Perl, MySQL UDF, etc.
Job: The request scheduler, used to coordinate the forwarding of requests sent by the Client to appropriate Work.
Worker: The request processor, which can be C, PHP, Perl, and so on.


Because the Client and Worker do not limit the use of the same language, it is conducive to the integration between multiple systems in multiple languages.


Even by adding more workers, we can easily implement the distributed load balancing architecture of applications.

2. install the Gearman job distribution server

Install Gearman server and library:
Wget http://launchpad.net/gearmand/trunk/0.8/+download/gearmand-0.8.tar.gz
Tar zxf gearmand-0.8.tar.gz
Cd gearmand-0.8
./Configure
Sudo make
Sudo make install

Some problems may occur in the middle:
In./configure, libraries may be missing. Generally, the libevent and uuid development kits are missing and installed...

Sudo apt-get install libevent-dev
Sudo apt-get install uuid-dev

After the installation is complete, re-configure the installation. after the installation is complete, execute
Sudo ldconfig

3. Gearman Client and Worker PHP implementation instance

Install Gearman PHP extension:
Wget http://pecl.php.net/get/gearman-0.4.0.tgz
Tar zxf gearman-0.4.0.tgz
Cd gearman-0.4.0
Phpize
./Configure
Sudo make
Sudo make install

Possible problems:
Phpize command not found, phpize in php development kit, so first install php5-dev

Sudo apt-get install php5-dev

After installation, you can execute phpize in the source code directory to generate the relevant installation configuration information, and then execute the./configure

After make install, it tells you a directory where the generated gearman. so is located.

Enter the extension directory of the corresponding PHP as needed (because I directly use the default php installed by the system, it is automatically generated in the extension)


Next, modify php. ini to make php load the module:
Php -- ini

Check where php. ini is, and modify sudo vim to add
Extension = "gearman. so"

Then, write the client and worker

Client. php

[Php]View plain copy

$ Client = new GearmanClient ();

$ Client-> addServer ("127.0.0.1", 4730 );

Print $ client-> do ("title", "Linvo ");

Print "/n ";

?>

Worker. php

[Php]View plain copy

$ Worker = new GearmanWorker ();

$ Worker-> addServer ("127.0.0.1", 4730 );

$ Worker-> addFunction ("title", "title_function ");

While (true ){

$ Worker-> work ();

If ($ this-> worker-> returnCode ()! = GEARMAN_SUCCESS ){
// Log or exception handling is required for Gearman status errors
}

}

Function title_function ($ job)

{

$ Str = $ job-> workload ();

Return strlen ($ str );

}

?>

The preparation has been completed and the Test starts.
1. start a job
Gearmand-d

2. start worker
Php-c/etc/php5/apache2/php. ini worker. php

3. start the client (open in new terminal)
Php-c/etc/php5/apache2/php. ini client. php

The length of the string displayed on the screen is "5"

Here, we need to describe the following points:
1. run the command directly in php cli. the-c parameter is added to load the php. ini configuration file to load the gearman extension.
2. worker should be implemented as a daemon (CLI mode). multiple worker nodes can be enabled, so that the tasks initiated by the client are distributed to each worker for execution (automatic load balancing)
This example is too simple to see the effect even if multiple workers are enabled. However, you can terminate one of them to see that the system automatically switches to another worker and continues to run normally.
3. Similarly, multiple clients can be enabled (for details about the model, refer to the previous logs)

4. you can also enable multiple jobs to avoid SPOF.