Distributing workloads for PHP applications with Gearman

Distributing workloads for PHP applications with Gearman
http://www.ibm.com/developerworks/cn/opensource/os-php-gearman/

Using Gearman from PHP

Using Gearman from PHP is similar to the previous example, the only difference being that this is the creation of producer and consumer within PHP. Each consumer work is encapsulated within one or more PHP functions.

Listing 1 shows a Gearman worker written in PHP. Save the code in a file named worker.php.


Listing 1. worker.php

$worker = new Gearmanworker ();
$worker->addserver ();
$worker->addfunction ("title", "Title_function");
while ($worker->work ());

function Title_function ($job)
{
Return Ucwords (Strtolower ($job->workload ()));
}
?>




Listing 2 shows a producer, or client, written in PHP. Save this code in a file named client.php.


Listing 2. client.php

$client = new Gearmanclient ();
$client->addserver ();
Print $client->do ("title", "All the world ' s STagE");
print "\ n";
?>




Now, you can connect the client and worker with the following command line:

$ PHP worker.php &
$ PHP client.php
All the world ' s A Stage
$ jobs
[3]+ Running php worker.php &




The worker application continues to run and is ready to serve another client.


--------------------------------------------

Back to top of page

Advanced Features of Gearman

Gearman may be used in many places within a WEB application. You can import large amounts of data, send many e-mails, encode video files, dig data, and build a central logging facility-all without compromising the experience and responsiveness of your site. Data can be processed in parallel. Also, because the Gearman protocol is language-and platform-independent, you can mix programming languages in your solution. For example, you can write a producer in PHP and write a worker in C, Ruby, or any other language that supports the Gearman library.

A Gearman network that connects the client to the worker can actually use any structure you can imagine. Many configurations can run multiple agents and assign workers to many machines. Load balancing is implicit: each actionable worker (possibly with more than one worker per worker host) pulls the job out of the queue. A job can run synchronously or asynchronously and has a priority.

The latest version of Gearman has extended the system features to include durable job queues and a new protocol to submit work requests over HTTP. For the former, the Gearman Task Force column is saved in memory and stored in a relational database for backup. This way, if the Gearman daemon fails, it can recreate the task queue after a reboot. Another recent improvement is to increase queue persistence through a memcached cluster. Memcached storage is also dependent on memory, but is dispersed over several machines to avoid a single point of failure.

Gearman is a just-starting but very powerful work distribution system. According to Gearman's author, Eric day, Yahoo! uses Gearman on 60 or more servers to process 6 million jobs daily. News aggregator Digg has built a Gearman network of the same size and can handle 400,000 jobs a day. A good example of Gearman can be found in Narada, an open source search engine (see Resources).

Future versions of Gearman will collect and report statistics, provide advanced monitoring and caching job results, and more. To track this Gearman project, you can subscribe to its Google group, or visit its IRC channel on the Freenode #gearman.

