Use Gearman to distribute the workload of PHP applications

Address: www. phpthinking. comarchives518 although most of the content of a Web application is related to the representation, its price and competitive advantage may be reflected in several proprietary services or algorithms. If such processing is too complex or delayed, it is best to perform asynchronous execution to prevent the Web server from responding to the incoming request. Actually

Address: http://www.phpthinking.com/archives/518 although most of the content of a Web application is associated with representation, its price and competitive advantage may be reflected in several proprietary services or algorithms. If such processing is too complex or delayed, it is best to perform asynchronous execution to prevent the Web server from responding to the incoming request. Actually

Address: http://www.phpthinking.com/archives/518

Although most of the content of a Web application is related to representation, its value and competitive advantage may be reflected in several proprietary services or algorithms. If such processing is too complex or delayed, it is best to perform asynchronous execution to prevent the Web server from responding to the incoming request. In fact, it is better to put a computing-intensive or specialized function on one or more independent dedicated servers.

Common acronyms

• API: Application Programming Interface
• HTTP: Hypertext Transfer Protocol
• LAMP: Linux, Apache, MySQL, and PHP

The PHP Gearman library can distribute work to a group of machines. Gearman queues jobs and distributes a small number of jobs to distribute complex tasks to the machines reserved for this task. This library is applicable to Perl, Ruby,CPython and PHP can be used by developers, and can also run on any UNIX-like? On the platform, including Mac OS X and Linux? And Sun Solaris.

It is very easy to add Gearman to a PHP application. If you host a PHP application in a typical LAMP configuration, Gearman requires an additional daemon and a PHP extension. By November 2009, the latest version of the Gearman Daemon was 0.10, and two PHP extensions were available-one wrapped in PHP with Gearman.CLibrary, the other is written in pure PHP. We need to use the former.

Note: For this article, producer refers to the machine that generates the work request, consumer refers to the machine that executes the work, and agent refers to the intermediary that connects the producer to the appropriate consumer.

Install Gearman

Adding a Gearman to a machine requires two steps: Step 1 to build and start the daemon, and step 2 to build a PHP extension that matches the PHP version. This daemon package includes all the libraries required to build this extension.

First, download the Gearman daemon.gearmandThe latest source code of, decompress this tarball, build and install this Code (the installation requires Super User Permissions, that is, root user permissions ).

$ wget http://launchpad.net/gearmand/trunk/\  0.10/+download/gearmand-0.10.tar.gz$ tar xvzf gearmand-0.10.tar.gz$ cd gearmand-0.10$ ./configure$ make$ sudo make install

InstallgearmandBuild PHP extension. You can obtain the tarball from PECL or copy the Repository from Github.

$ wget http://pecl.php.net/get/gearman-0.6.0.tgz$ cd pecl-gearman## or#$ git clone git://github.com/php/pecl-gearman.git$ cd pecl-gearman

With the code, you can start building extensions:

$ phpize$ ./configure$ make$ sudo make install

This Gearman daemon is usually installed in/usr/sbin. You can directly start the daemon from the command line or add the daemon to the startup configuration so that the daemon can be started every time the machine restarts.

Next, install the Gearman extension. Open the php. ini file (you can usephp --iniCommand to quickly find the file), and then add the code lineextension = gearman.so:

$ php --iniLoaded Configuration File:         /etc/php/php.ini$ vi /etc/php/php.ini ...extension = gearman.so

Save the file. To verify whether the extension is enabled, runphp --infoAnd then find Gearman:

$ php --info | grep "gearman support"gearmangearman support => enabledlibgearman version => 0.10

In addition, you can use a PHP code snippet to verify that the build and installation are appropriate. Save this small application to verify_gearman.php:

Next, run the program from the command line:

$ php verify_gearman.php0.10

If the version number matches the version number of the previously built and installed Gearman library, the system is ready.

Run Gearman

As we mentioned earlier, a Gearman configuration has three roles:

• One or more producers generate work requests. Each work Request Name the function it wants, for exampleemail_allOranalyze.
• One or more consumers complete the request. Each consumer name one or more functions it provides and registers these functions with the agent. A consumer can also be called a worker.
• The proxy centrally compiles all the services provided by the consumer with which the connection is established. It associates the producer with the appropriate consumer.

With the following command line, you can immediately experience Gearman:

1. Start this agent, that is, the Gearman daemon:

   $ sudo /usr/sbin/gearmand --daemon

2. Use the command line utilitygearmanRun a worker. This worker requires a name and can run any command line utility. For example, you can create a worker to list the contents of a directory.-fThe parameter is named as the function provided by the worker:

   $ gearman -w -f ls -- ls -lh

3. The last code block is a producer or a job used to generate a query request. You can also usegearmanGenerate a request. Similarly-fTo specify the service from which you want to get help:

   $ gearman -f ls < /dev/nulldrwxr-xr-x@ 43 supergiantrobot  staff   1.4K Nov 15 15:07 gearman-0.6.0-rw-r--r--@  1 supergiantrobot  staff    29K Oct  1 04:44 gearman-0.6.0.tgz-rw-r--r--@  1 supergiantrobot  staff   5.8K Nov 15 15:32 gearman.htmldrwxr-xr-x@ 32 supergiantrobot  staff   1.1K Nov 15 14:04 gearmand-0.10-rw-r--r--@  1 supergiantrobot  staff   5.3K Jan  1  1970 package.xmldrwxr-xr-x  47 supergiantrobot  staff   1.6K Nov 15 14:45 pecl-gearman

Use Gearman from PHP

Using Gearman in PHP is similar to the previous example. The only difference is that producer and consumer are created in PHP. The work of each consumer is encapsulated in one or more PHP functions.

Listing 1 shows a Gearman worker written in PHP. Save the code in a file named worker. php.

Listing 1. Worker. php

 addServer();  $worker->addFunction("title", "title_function");  while ($worker->work());     function title_function($job)  {    return ucwords(strtolower($job->workload()));  }?>

Listing 2 shows a producer or client written in PHP. Save the code in a file named client. php.

Listing 2. Client. php

 addServer();  print $client->do("title", "AlL THE World's a sTagE");  print "\n";?>

Now, you can use the following command line to connect the client to the worker:

$ php worker.php &$ php client.phpAll The World's A Stage$ jobs[3]+  Running                 php worker.php &

This worker application continues to run and is ready to serve another client.

Advanced features of Gearman

Gearman may be used in many places in a Web application. You can import large amounts of data, send many emails, encode video files, dig data, and build a central logging facility-all of which do not affect the site experience and responsiveness. Data can be processed in parallel. Furthermore, since the Gearman protocol is independent of languages and platforms, you can use a hybrid programming language in the solution. For example, you can use PHP to compile a producer.C, Ruby, or any other language that supports the Gearman library to write worker.

A Gearman network connected to the client and worker can actually use any structure you can imagine. Many configurations can run multiple proxies and allocate worker to many machines. Load Balancing is implicit: Each operable available worker (may be that each worker host has multiple workers) pulls jobs from the queue. A job can run synchronously or asynchronously and has a priority.

The latest version of Gearman has extended system features to include persistent job queues and used a new Protocol to submit work requests over HTTP. For the former, the Gearman work queue is stored in the memory and backed up in the memory of a relational database. In this way, if the Gearman daemon fails, it can re-create the working queue after restart. Another latest improvement adds queue persistence through a memcached cluster. Memcached storage also relies on memory, but is dispersed on several machines to avoid spof.

Gearman is a very powerful work distribution system that has just started. According to Eric Day, author of Gearman, Yahoo! Use Gearman to process 6 million jobs per day on 60 or more servers. The news aggregator Digg has built a Gearman network of the same size and can process 400,000 jobs every day.