Use of Gearman

Brief introduction

Gearman provides a common program framework for distributing your tasks to different machines or different processes. It provides you with the ability to work in parallel, load balancing, and the ability to communicate between different programming languages. Gearman can be used in a wide range of areas, from highly available Web sites to database replication tasks. In short, Gearman is the central system responsible for distributing processing, and its advantages include

Open Source: Gearman is free and open source and has a very active open source community, if you want to make some contributions, please click. Multi-lingual Support: Gearman supports a wide variety of languages. Let's write a worker program in one language, but write a client program in a different language. Flexible: Don't be stuck in a fixed form. You can take any form you want, such as Map/reduce. Fast: Gearman protocol is very simple, and has a C language implementation of the optimized server, to ensure that the application load at a very low level. Implantable: Because the Gearman is very small and flexible. So you can put him into any existing system. No single point: Gearman not only helps to extend the system, but also avoids system failures.

How the Gearman works

Applications that use Gearman typically consist of three parts: a client, a worker, and a task server. The role of the client is to present a job task to the job server task servers. The job Server will look for a suitable Worker to complete the task. The Worker executes the job sent by the client and returns the result to the client through the job Server. Gearman provides the Client and Worker APIs that enable the use of these API applications to communicate with the Gearman Job server. Communication between the Gearman internal Client and Worker is done through a TCP connection. The process of working as shown

The usefulness of Gearman

Gearman First provides a multi-lingual communication interface, of course, there is a simpler and more effective way. Gearman can share the load of work to different machines, as shown in

Installing Gearman

Adding Gearman to a machine takes two steps: The first step is to build and launch the daemon, and the second step is to build a PHP extension that matches the PHP version. This daemon package includes all the libraries needed to build this extension

Install GCC

Yum Install gcc-c++

Install boost

wget https://github.com/boostorg/build/archive/2014.10.tar.gz./bootstrap.sh./b2 Install

Installing Qperf

Yum Install Gperf

Installing Libevent Libevent-devel

Yum-y Install libevent Libevent-devel

Installing Libuuid

wget Http://downloads.sourceforge.net/project/libuuid/libuuid-1.0.3.tar.gztar ZXVF libuuid-1.0.3.tar.gz./ Configure Make & make install

Installing Gearmand

wget Https://launchpad.net/gearmand/1.2/1.1.11/+download/gearmand-1.1.11.tar.gztar ZXVF GEARMAND-1.1.11.TAR.GZCD Gearmand-1.1.11./configuremake & make Install

Installing PHP extensions for Gearman

wget Http://pecl.php.net/get/gearman-1.1.2.tgzphpize./configuremake & make Install

Start the Job Server

Gearmand-d[[email protected] www]# Ps-ef | grep gearmandroot     19109     1  0 01:06?        00:00:00/usr/local/sbin/gearmand-droot     22185  3160  0 03:19 pts/1    00:00:00 grep gearmand

Command line use

Ldconfig [worker][[email protected] www] gearman-w-f WC--wc-l[client][[email protected] www] gearman-f WC </etc/ Passwd32

PHP script uses

[Worker.php]<?php$worker = new Gearmanworker (); $worker->addserver (); $worker->addfunction ("Strtoupper", " My_func "), while ($worker->work ()), function My_func ($job) {    return Strtoupper ($job->workload ());} [client.php]<?php$client = new Gearmanclient (); $client->addserver (); $string = $client->donormal (" Strtoupper "," Hello World "); Echo $string; Implementation [Email protected] www]php worker.php[[email protected] www]php Client.phphello World

Gearmand using Mysql as a queue store

[start]gearmand-d-q mysql--mysql-host=localhost--mysql-port=3306--mysql-user= Root \--mysql-password=123456--mysql-db=test--mysql-table=gearman_queue[view datasheet]mysql> show tables;+------------ -------+| Tables_in_gearman |+-------------------+| Gearman_queue |+-------------------+1 row in Set (0.00 sec) [Test] #addTask: gearman-f testqueue-b xx00mysql> Select * FROM gearman_queue;+--------------------------------------+---------------+----------+------+-------------+| Unique_key | function_name | Priority | Data | When_to_run |+--------------------------------------+---------------+----------+------+-------------+| 75f18bac-2f95-11e5-87c3-cb400fa24a6b |        Testqueue | 1 |           xx00 | 0 |+--------------------------------------+---------------+----------+------+-------------+1 row in Set (0.00 sec) # Dojob:gearman-f testqueue-wmysql> SELECT * from Gearman_queue; Empty Set (0.00 sec) 

Advanced Features of Gearman

Gearman may be used in many places within a WEB application. You can import large amounts of data, send many e-mails, encode video files, dig data, and build a central logging facility-all without compromising the experience and responsiveness of your site. Data can be processed in parallel. Also, because the Gearman protocol is language-and platform-independent, you can mix programming languages in your solution. For example, you can write a producer in PHP with a C worker in Ruby or any other language that supports the Gearman library.

A Gearman network that connects the client to the worker can actually use any structure you can imagine. Many configurations can run multiple agents and assign workers to many machines. Load balancing is implicit: each actionable worker (possibly with more than one worker per worker host) pulls the job out of the queue. A job can run synchronously or asynchronously and has a priority.

The latest version of Gearman has extended the system features to include durable job queues and a new protocol to submit work requests over HTTP. For the former, the Gearman Task Force column is saved in memory and stored in a relational database for backup. This way, if the Gearman daemon fails, it can recreate the task queue after a reboot. Another recent improvement is to increase queue persistence through a memcached cluster. Memcached storage is also dependent on memory, but is dispersed over several machines to avoid a single point of failure.

Gearman is a just-starting but very powerful work distribution system. According to Gearman's author, Eric day, Yahoo! uses Gearman on 60 or more servers to process 6 million jobs daily. News aggregator Digg has built a Gearman network of the same size and can handle 400,000 jobs a day. A good example of Gearman can be found in Narada, an open source search engine (see Resources).

References

http://www.ibm.com/developerworks/cn/opensource/os-php-gearman/

Http://www.cnblogs.com/cocowool/archive/2011/08/18/2145144.html

Use of Gearman