Gearman installation, test notes

Official Website: http://gearman.org/

Gearman-0.7.0.tgz php extensions
Gearman-mysql-udf-0.5.tar.gz
Gearmand-0.14.tar.gz c edition server [and java, perl edition]
Yugene-Gearman-Monitor-ed34347.zip monitoring
Https://github.com/yugene/Gearman-Monitor monitoring

Http://baike.baidu.com/view/2926980.htm
Http://www.cnblogs.com/cocowool/archive/2011/08/18/2145144.html

Deploy Gearman across multiple Environments
Http://www.ibm.com/developerworks/cn/opensource/os-gearman/index.html
Build a distributed image processing platform using open-source Gearman framework-Zhang Yan
Http://blog.s135.com/dips/

Gearman is a machine used to delegate work to other machines and distributed calls. It is more suitable for machines that do a job, concurrent jobs, load balancing among multiple calls, or in the system that calls functions in other languages.
Gearman provides a general program framework to distribute your tasks to different machines or processes. It provides you with the ability to perform parallel work, Server Load balancer processing, and communication between different programming languages. Gearman can be used in a wide range of fields, from highly available websites to database replication tasks. In short, Gearman is the central system responsible for distribution and processing. Its advantages include:
Open-source: Gearman is free and open-source and has a very active open-source community. If you want to make some contributions, click.
Multi-language support: Gearman supports a wide variety of languages. This allows us to write a Worker program in one language, but a Client program in another language.
Flexibility: Do not stick to a fixed form. You can use any form you want, such as Map/Reduce.
Fast: The Gearman protocol is very simple, and there is a server that is implemented in C language and optimized to ensure that the application load is very low.
Deployable: Because Gearman is very small and flexible. Therefore, you can place it in any existing system.
No spof: Gearman not only helps to expand the system, but also avoids system failures.

Directory

Introduction to Gearman
Gearman running process
Introduction to the new version of Gearman
Install Gearman
Expand

Introduction to Gearman
Generally, multi-language and multi-system integration is a big problem. Generally, most people use WebService to solve such integration problems. However, no matter what type of WebService, such as RPC, or REST style. In contrast, Gearman can also achieve similar functions and is easier to use.
Gearman running process
The processing of a Gearman request involves three roles: Client-> Job-> Worker.
Client: The request initiator, which can be C, PHP, Perl, MySQL UDF, etc.
Job: The request scheduler, used to coordinate the forwarding of requests sent by the Client to appropriate Work.
Worker: The request processor, which can be C, PHP, Perl, and so on.
Because the Client and Worker do not limit the use of the same language, it is conducive to the integration between multiple systems in multiple languages.
Even by adding more workers, we can easily implement the Distributed Load Balancing architecture of applications.
Introduction to the new version of Gearman
Gearman released version January 16 in c in 0.2. gearman provides a framework for transferring work to other machines or calling other machine functions. With gearman, you can achieve parallel work and load balancing, call functions in other languages. It can be applied in many scenarios, from highly available web services to database replication. In other words, it is the central nervous system of distributed processing. Gearman was developed in the perl Language Implemented by sixpart in the early days, and its name came from "Manager". This framework is only responsible for distributing the work to be done and does not do any practical work. (This is the same as spread toolkit ).
Install Gearman
Install Gearman server and library
Tar zxf gearmand-0.8.tar.gz [1]
Cd gearmand-0.8
./Configure
Make
Make install
Install Gearman PHP extension
Tar zxf gearman-0.4.0.tgz
Cd gearman-0.4.0
Phpize
./Configure
Make
Make install
Edit the php. ini configuration file to load the corresponding module and make it take effect:
Extension = "gearman. so"

/Www/wdlinux/php/bin/phpize
./Configure -- with-php-config =/www/wdlinux/php/bin/php-config
Make & make install

/Www/wdlinux/php/bin/php -- info | grep gearman
2. In another terminal, change to this source directory and run:

/Www/wdlinux/php/bin/php examples/reverse_worker.php

3. In another terminal, change to this source directory and run:

/Www/wdlinux/php/bin/php examples/reverse_client.php

Start Job:
Gearmand-d
If the current user is root, perform the following operations:
Gearmand-d-u root
Port 4730 is used by default, which will be used below.
Note: If you cannot find the path of the gearmand command, do not forget to use whereis gearmand for confirmation. Gearmand:/usr/local/sbin/gearmand

Start with a daemon:
Gearmand-L 192.168.0.1-p 4730-u root-d

Compile Worker
The content of the worker. php file is as follows:
<? Php
$ Worker = new GearmanWorker ();
$ Worker-> addServer ('127. 0.0.1 ', 127 );
$ Worker-> addFunction ('reverse', 'My _ reverse_function ');
While ($ worker-> work ());
Function my_reverse_function ($ job ){
Return strrev ($ job-> workload ());
}
?>
Set the background to run work
Php worker. php &
Write Client
The content of the client. php file is as follows:
<? Php
$ Client = new GearmanClient ();
$ Client-> addServer ('127. 0.0.1 ', 127 );
Echo $ client-> do ('reverse', 'Hello World! '), "\ N ";
?>
Run client
Php client. php
Output :! DlroW olleH
For convenience, the Worker and Client Use PHP, but this does not affect the demonstration. in actual application, you can use Gearman to integrate Worker and Client implemented in different languages. At this point, you may want to know the Server Load balancer function mentioned above: it is very simple. You only need to add multiple workers, you can follow the Worker. php writes a few more similar files and sets different return values to identify the demo effect. Then start these Worker files in turn and use client. php for multiple requests. Then you will find that the Job will forward the Client requests to different workers.
Command line tool
If you think installing PHP is too troublesome, you can use the command line tool to experience the features of Gearman:
Start Worker: gearman-w-f wc -- wc-l &
Run Client: gearman-f wc </etc/passwd

-- I haven't encountered any accidents.
1. the source code of the libs/program_options directory must be compiled or referenced.
2. Use g ++-o simple. cpp-lboost_program_options-L/usr/local/lib to compile the program.
Http://www.cnblogs.com/cocowool/archive/2011/08/18/2144142.html
Install Boost C ++ Libraries on Linux
Gearman depends on the program_options library in Boost C ++ Libraries, and the required version is later than 1.39. The self-contained 5.2 In the CentOS 1.33 system cannot meet the requirements. Some problems may occur during the installation process. We recommend that you refer to the Easy Build and Install section in the Boost official manual.

After Gearman is installed, the error while loading shared libraries: libgearman. so.1 may occur when it is enabled for the first time. You can use the following method to solve the problem:
Error while loading shared libraries: libgearman. so.1 occurs when/usr/local/sbin/gearmand-d is run.
Run/sbin/ldconfig-v and then run gearman.

The ldconfig command is mainly used in the default search directory (/lib and/usr/lib) and dynamic library configuration file/etc/ld. so. find the shared dynamic link library under the directory listed in conf (format as described earlier, lib *. so *) to create a dynamic loader (ld. so) the required connection and cache file. the default cached file is/etc/ld. so. cache. This file stores the list of Dynamic Linked Library names in the sorted order.
Ldconfig is usually run when the system starts. When you install a new dynamic link library, you need to manually run this command.

One thing to be careful about is data sharing. Gearman does not perform any conversion or operation on the exchanged data. There is no problem with the simple strings and integers used here, but the array values in PHP cannot be shared and expected to be understood in Java. For this type of interaction, you can use one of many structured data standards, such as JavaScript Object Notation (JSON) or XML. In addition, if you want to share the ID or find the information you want to use when processing the information from the database, or use a transparent method such as memcached (although JSON or equivalents may still be required ).

-------------
Test-z "/usr/local/lib/pkgconfig" |/bin/mkdir-p "/usr/local/lib/pkgconfig"
/Usr/bin/install-c-m 644 support/gearmand. pc'/usr/local/lib/pkgconfig'

Installing shared extensions:/www/wdlinux/php-5.2.17/lib/php/extensions/no-debug-non-zts-20060613/

/Www/wdlinux/php/bin/php examples/reverse_worker.php [run multiple tasks to achieve load]
/Www/wdlinux/php/bin/php examples/reverse_client.php

1. Pass the value a name and a serialized parameter.

Advanced features of Gearman
Gearman may be used in many places in a Web application. You can import large amounts of data, send many emails, encode video files, dig data, and build a central logging facility-all of which do not affect the site experience and responsiveness. Data can be processed in parallel. Furthermore, since the Gearman protocol is independent of languages and platforms, you can use a hybrid programming language in the solution. For example, you can write a producer in PHP and worker in C, Ruby or any other language that supports the Gearman library.

A Gearman network connected to the client and worker can actually use any structure you can imagine. Many configurations can run multiple proxies and allocate worker to many machines. Load Balancing is implicit: Each operable available worker (may be that each worker host has multiple workers) pulls jobs from the queue. A job can run synchronously or asynchronously and has a priority.

The latest version of Gearman has extended system features to include persistent job queues and used a new Protocol to submit work requests over HTTP. For the former, the Gearman work queue is stored in the memory and backed up in the memory of a relational database. In this way, if the Gearman daemon fails, it can re-create the working queue after restart. Another latest improvement adds queue persistence through a memcached cluster. Memcached storage also relies on memory, but is dispersed on several machines to avoid spof.

Gearman is a very powerful work distribution system that has just started. According to Eric Day, author of Gearman, Yahoo! Use Gearman to process 6 million jobs per day on 60 or more servers. The news aggregator Digg has built a Gearman network of the same size and can process 400,000 jobs every day. An outstanding example of Gearman can be found in the open source search engine Narada (see references.