PHP uses the php-resque library and Redis to implement the MQ tutorial. php-resqueredis

PHP uses the php-resque library and Redis to implement the MQ tutorial. php-resqueredis

Problems caused by background tasks processed by message queues
Projects often have the need to run tasks in the background. For example, when sending an email, it may take 5 to 10 seconds or longer to connect to the email server. If you can give the user a success prompt, then, the process of sending emails is processed slowly in the background, which obviously provides a better user experience.

To achieve similar requirements, the general implementation method in Web projects is to use Message Queue (Message Queue), such as MemcacheQ and RabbitMQ, which are all well-known products.

To put it bluntly, a message queue is the simplest first-in-first-out queue, and a member of the queue is a piece of text. It is precisely because the message queue is too simple. When taking the message queue, it is a bit difficult to start with, because this is only a task for sending emails, it will lead to many problems:

• A Message Queue can only store string-type data. How can I convert a "task" such as sending an email to a "message" in the message queue "?
• The message queue is only responsible for data storage and entry and exit, and itself cannot execute any program. Therefore, we need to retrieve data from the Message Queue one by one, and then convert the data back to the task for execution.
• We cannot predict when data will be generated in the message queue. Therefore, our task execution program also needs to be able to monitor the message queue, that is, a daemon process resident in the background.
• Generally, PHP runs in cgi Mode and cannot be resident in memory. We know that php also has the cli mode. Can the daemon process be implemented using php cli? How is the efficiency?
• When the daemon is running, can the Web application interact with the background daemon to enable/kill the process and obtain the running status of the process?

Resque's design of background tasks and division of roles

For the above questions, the best answer I can find so far is not from php, but from Ruby's Resque project. It is precisely because Resque clearly and simply solves a series of problems brought about by background tasks, the Resque design has also been cloned to Python, php, NodeJs, and other languages: for example, pyres in Python and PHP-resque in php. Here there are various language versions of Resque implementation, in this log, we will take the PHP version as an example to illustrate how to use php-resque to run a background task. Some details may differ from the Ruby version, however, this article uses the php version as the standard.

Resque solves these problems as follows:

Background task role Division
In fact, from the above problems, we can see that only one message queue can not solve all the problems, and new roles need to be involved. In Resque, a background task is abstracted to be completed by three roles:

• Job | task: a Job is a task that needs to be completed in the background. For example, a Job can be abstracted as a Job by sending an email as an example in this article. In Resque, a Job is a Class.
• Queue | Queue: The Message Queue above. In Resque, the Queue is implemented by Redis. Resque also provides a simple queue manager that can insert/remove jobs to/from queues.
• Worker | executor: obtains and executes jobs from the queue, and can run in the background as a daemon.

Based on this division, the basic process of a background task under Resque is as follows:

• Compile a background task into an independent Class, which is a Job.
• Where you need to use the background program, the system puts the Job Class name and required parameters into the queue.
• Start a Worker in the command line mode and specify the queue to be processed by the Worker through parameters.
• Worker runs as a daemon and regularly checks the queue.
• When there is a Job in the queue, the Worker extracts the Job and runs it, that is, instantiates the Job Class and executes the methods in the Class.

This completes a background task.

There is also an important design in Resque: a Worker can process one queue or many queues, in addition, you can increase the number of Worker processes/threads to speed up queue execution.

Php-resque Installation
It should be noted in advance that, due to the process development and management involved, php-resque uses the PCNTL function of php, so it can only run in Linux and requires php to compile the PCNTL function. If you want to do the same job in Windows, you can look for other Resque language versions. php is not suitable for background tasks in Windows.

Taking Ubuntu12.04LTS as an example, the PCNTL function has been compiled by default in php installed with apt in Ubuntu without any configuration. The following commands are root accounts.

Install Redis

apt-get install redis-server

Install Composer

apt-get install curlcd /usr/local/bincurl -s http://getcomposer.org/installer | phpchmod a+x composer.pharalias composer='/usr/local/bin/composer.phar'

Use Composer to install php-resque
Assume that the web directory is in/opt/htdocs

apt-get install git git-corecd /opt/htdocsgit clone git://github.com/chrisboulton/php-resque.gitcd php-resquecomposer install

Use of php-resque
1. Write a Worker
In fact, php-resque has provided a simple example. The demo/job. php file is the simplest Job:

class PHP_Job{  public function perform()  {    sleep(120);    fwrite(STDOUT, 'Hello!');  }}

The Job outputs the Hello character to STDOUT in 120 seconds!

In the Resque design, a Job must have a perform method, and the Worker will automatically run this method.

2. Insert a Job into a queue
Php-resque also provides the simplest queue insertion implementation demo/queue. php:

if(empty($argv[1])) {  die('Specify the name of a job to add. e.g, php queue.php PHP_Job');}require __DIR__ . '/init.php';date_default_timezone_set('GMT');Resque::setBackend('127.0.0.1:6379');$args = array(  'time' => time(),  'array' => array(    'test' => 'test',  ),);$jobId = Resque::enqueue('default', $argv[1], $args, true);echo "Queued job ".$jobId."\n\n";

In this example, queue. php needs to run in cli mode. It inserts the first parameter received by cli as the Job name into the queue named 'default' and outputs the Job Id of the queue just inserted to the screen. Enter at the terminal:

php demo/queue.php PHP_Job

The result shows the output on the screen:

Queued job b1f01038e5e833d24b46271a0e31f6d6

That is, the Job has been added successfully. Note that the Job name here is consistent with the name of the Job Class we wrote: PHP_Job

3. view the Job running status
Php-resque also provides an example of viewing the Job running status, run directly:

php demo/check_status.php b1f01038e5e833d24b46271a0e31f6d6

The output is as follows:

Tracking status of b1f01038e5e833d24b46271a0e31f6d6. Press [break] to stop. Status of b1f01038e5e833d24b46271a0e31f6d6 is: 1

The status of the Job we just created is 1. In Resque, a Job has the following four States:

• Resque_Job_Status: STATUS_WAITING = 1; (waiting)
• Resque_Job_Status: STATUS_RUNNING = 2; (in progress)
• Resque_Job_Status: STATUS_FAILED = 3; (failed)
• Resque_Job_Status: STATUS_COMPLETE = 4; (ended)

Because no Worker is running, the created Job is still waiting.

4. Run Worker
This time, we will directly compile demo/resque. php:

<?phpdate_default_timezone_set('GMT');require 'job.php';require '../bin/resque';

You can see that a Worker requires at least two parts:

You can directly include a Job file or use the php automatic loading mechanism to specify the path of the Job Class and perform automatic loading.
Default Worker including Resque: bin/resque
Run in the terminal:

QUEUE=default php demo/resque.php

The preceding QUEUE part sets environment variables. we specify that the current Worker is only responsible for processing the default QUEUE. You can also use

QUEUE=* php demo/resque.php

To process all queues.

After running, the output is

#!/usr/bin/env php*** Starting worker

Run the ps command to check:

ps aux | grep resque

A php daemon is running.

1000   4607 0.0 0.1 74816 11612 pts/3  S+  14:52  0:00 php demo/resque.php

Then use the previous check Job command

php demo/check_status.php b1f01038e5e833d24b46271a0e31f6d6

2 minutes later.

Status of b1f01038e5e833d24b46271a0e31f6d6 is: 4

The task has been run, and the output Hello is displayed on the screen!