Deploy Gearman across multiple environments

Brief introduction

One of the greatest challenges in modern computing environments is the distribution of work and the efficient use of computational resources. The general trend is that it is cheap and easy to install a powerful machine to perform more intuitive and simple tasks, but this may not get the best overall performance and the best use of the machine. Instead, many applications now find themselves needing to perform some fairly small operations, but may have to perform thousands of times without a powerful machine.

As the field of calculation (farm) increases, it is sometimes necessary to perform an action or operation across all machines, or to selectively perform administrative or installation tasks.

There are a number of solutions available. For example, virtualization is one of the ways in which computers are fragmented for optimal performance, although they have their own problems and limitations. The Memcached tool (which uses standby memory) and a grid-like solution--IBM Cloud also supports the flexible allocation of computing power, allowing computing power to be used everywhere.

Gearman uses a different approach, which provides a flexible mechanism for distributing discrete tasks across a set of machines. This approach enables the expansion and more efficient utilization of the system while significantly reducing the complexity of maintaining and supporting such systems. This has spawned many solutions, including a grid environment, and even a Web service base, but all use a practical distribution approach. This makes Gearman useful in a grid environment or in a WEB environment where you want to share and distribute work and requests. It can also be used to unload tasks and process information from the Web, database, and Amazon environments, making it ideal for supporting IBM websphere® or Amazon deployments.

Gearman Foundation

The Gearman service has many elements that make it not just a way to commit and share work, but the main system consists of only three components: the Gearmand daemon (or job server), the client that submits the request to the Gearman service, and the worker who performs the actual work (see Figure 1).

Figure 1. Gearman Service Overview

The Gearmand server performs a simple function of collecting work requests from the client and acting as a registrar (where workers can submit information about the types of work and operations they support). In a traditional grid service, a grid node is assigned by the Management Server to a specific task and a given job, which differs in Gearman from the worker's request for work.

This important distinction means that the decision on which worker is to do the work is not necessary. When distributing work, there are no configurations, priorities, or other organizations that are needed. Instead, the system is characterized by self-running and self managed, and for many installations, the basic structure that is initially described here is not complex.

Examples of Gearman

The easiest way to find out how Gearman works is to try a simple application that uses the Gearman service. You will use Perl for the workers and client elements in this example, but it is important to point out that you can also mix and match multiple languages to give full play to the advantages of a particular environment.

First, you must let the Gearmand server run. To do this, you can download the source code, run the configuration script, and run the server; you need to use the Libevent library. Many linux® distributions also have Gearman components that perform the installation without manual processing.

After the installation is complete, read the service using the-VV command-line option to find out what the server is doing when you start using it.

For client and worker components, you need to install the Gearman CPAN module, which includes the gearman::client and Gearman::worker modules, which are the modules you need to create clients and workers.

For a simple worker, we provide a basic word count function.

Listing 1. The Simple Gearman Worker

use　Gearman::Worker;
my　$worker　=　Gearman::Worker->new;
$worker->job_servers('192.168.0.2:4730');
$worker->register_function('wordcount'　=>　\&wordcount);
$worker->work　while　1;

sub　wordcount　
{
　　my　($input)　=　@_;

　　my　@words　=　split　/\s+/,$input->arg;

　　return　(scalar　@words);
}