Distributed Task Distribution framework-Gearman

Source: Internet
Author: User

Distributed Task Distribution framework-Gearman

Official documentation: http://gearman.org/getting-started/ installation methods and examples are available, you can take a look.

Gearman is a program framework for task distribution. It can be used in various scenarios. Compared with Hadoop, Gearman prefers task distribution. Its task distribution is very simple, which can be simply completed by using scripts. Gearman was originally used for the image resize function of LiveJournal. Because the image resize consumes a lot of computing resources, it needs to be scheduled to run on multiple backend servers. After the task is completed, it returns to the front end and then presents it to the interface.

What can Gearman do?

  • Asynchronous processing: image processing, order processing, batch mail/notification, etc.
  • High CPU or memory processing: large-capacity data processing, MapReduce operations, log aggregation, and video encoding
  • Distributed and Parallel Processing
  • Scheduled processing: incremental update and Data Replication
  • Maximum rate of FIFO Processing
  • Distributed System monitoring tasks

1. Installation Method

# Install gearmanyum install gearmand # install libgearmanyum install libgearman-devel # install gearman php extension pecl install gearman # Add gearman. so to php. iniecho "extension = gearman. so ">/etc/php. ini

Install it without pecl

yum install php-pear

If pecl cannot be installed, you can directly download the source code and use phpize to compile and install it.

 
cd ~/softwarewget http://pecl.php.net/get/gearman-1.1.2.tgzcd gearman-1.1.2phpize./configuremake && make install 

2. Simple Example: (we use Gearman to asynchronously process Baidu cloud PUSH Service)

First check the client. php registration event, client. php

<? Php // create a Gearman object $ client = new GearmanClient (); // The default value of addServer is localhost and the default value of port is 4730. If not, adjust $ client-> addServer (); // $ client-> addServer ("192.168.0.0", 4730); echo "Sending job \ n"; $ username = "test"; $ message = "message "; $ data = array ("username" => $ username, "message" => $ message,); // register events and PASS Parameters, multiple parameters can be converted using json_encode // the task can be run in a blocking manner, and the priority can be specified; of course, it can also be non-blocking (running does not wait for the result) run // you can refer to php Gearman api documentation: doNornal , DoHigh, doLow, doBackground $ result = $ client-> doBackground ("testFunction", json_encode ($ data); if ($ result) {echo "Success: $ result \ n ";}?>

Let's look at worker's processing. In fact, it is to register the event processing function, worker. php

<? Php $ worker = new GearmanWorker (); $ worker-> addServer (); // registers events and event processing functions $ worker-> addFunction ("testFunction", "handler "); // run workerwhile ($ worker-> work (); function handler (GearmanJob $ job) {$ workload = json_decode ($ job-> workload (); echo "received: ". print_r ($ workload, 1) ;}?>

You can run php client on the command line. php, php worker. php. it is best to start worker first to handle the event in a timely manner. You can also start the client first to queue the event in Gearman and wait for worker to process it;

You can also start multiple workers. Gearman automatically performs load balancing and assigns it to different workers for processing.

You can explore more application scenarios...

For example, use Gearman to synchronize data from MySQL to Redis (asynchronous replication)

Gearman monitoring:

You can use a supervisor or gearman manager.

For examples of using supervisor to monitor Gearman tasks, see:

This article permanently updates the link address:

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: info-contact@alibabacloud.com and provide relevant evidence. A staff member will contact you within 5 working days.

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.