Using PHP to implement daemon tasks background running and multithreading (Php-resque instructions)

Issues with Message Queuing for background tasks

Projects often have background running task requirements, such as sending mail, because to connect to the mail server, often need 5-10 seconds or more, if you can first give users a successful prompt message, and then slowly in the background to handle the operation of sending mail, obviously there will be a better user experience.

In order to achieve similar requirements, the general implementation of Web projects is to use Message Queuing, such as MEMCACHEQ,RABBITMQ, and so on, are well-known products.

Message Queuing It is a simple first-in-one queue in which a member of a queue is a piece of text. It is because the message queue is too simple, when holding the message queue, but a little bit of a feeling, because this is only a task to send the mail, there will be a lot of problems:

1. Message Queuing can only store data of a string type, how can a "task" such as sending a message be converted to a message in a message queue?

2. Message Queuing is only responsible for data storage and access, itself can not execute any program, then we want to retrieve the data from the message queue, then convert the data back to the task and execute.

3. We cannot predict when the message queue will have data generation, so our task execution program also needs to have the ability to monitor the message queue, which is a daemon that is resident in the background.

4. The General Web application PHP is running in CGI mode and cannot reside in memory. We know that PHP also has a CLI mode, then whether the daemon can be implemented in the PHP CLI, how efficient?

5. When the daemon runs, can web apps interact with the daemon, implement the ability to turn on/kill processes, and get the running state of the process?

Resque Design and Role division of background tasks

For these questions, the best answer I can find so far is not from PHP, but from the project Resque of Ruby, because Resque clearly and simply solves a series of problems caused by background tasks, Resque's design is also clone to Python, PHP, Nodejs and other languages: such as Python under the pyres and PHP php-resque, and so on, there are various language versions of the Resque implementation, and in this blog, of course, we have to use the PHP version as an example to show how to run a background task with Php-resque, There may be some discrepancies between the details and the Ruby version, but the PHP version will prevail in this article.

This is how Resque solves these problems:

Role partitioning for background tasks

In fact, from the above problem can be seen, only one message queue is unable to solve all problems, need new role intervention. In Resque, a background task is abstracted as a combination of three roles:

• Job | Task: A job is a task that needs to be done in the background, such as sending an e-mail for example in this article, it can be abstracted as a job. In Resque, a job is a class.

• Queue | Queue: That is, the message queue above, in Resque, the queue is implemented by Redis. Resque also provides a simple queue manager that enables functions such as inserting/removing jobs into the queue.

• Worker | Performer: Responsible for removing the job from the queue and executing it, which can be run in the background as a daemon.

So based on this division, a background task under the Resque of the basic process is this:

1. Write a background task as a separate class, which is a job.

2. Where background programs are required, the system places the name of the job class and the required parameters into the queue.

3. Open a worker as a command line and specify the queue that the worker needs to process by using parameters.

4. The worker runs as a daemon and checks the queue regularly.

5. When there is a job in the queue, the worker takes out the job and runs, instantiating the job class and executing the method in class.

At this point, you can run a full background task.

In Resque, there is also an important design: A worker that can handle a queue, can handle many queues, and can speed up the execution of a queue by increasing the number of worker processes/threads.

Installation of Php-resque

It should be explained in advance that, due to the opening and management of the process, Php-resque uses PHP's pcntl function, so it can only run under Linux and requires PHP to compile the PCNTL function. If you want to do the same with Windows, you can look for other language versions of Resque, and PHP is very unsuitable for background tasks under Windows.

Take Ubuntu12.04lts as an example, Ubuntu installed with apt PHP has been compiled by default pcntl function, without any configuration, the following instructions are the root account

Installing Redis

Apt-get Install Redis-server

Installing composer

Apt-get Install curlcd/usr/local/bincurl-s Http://getcomposer.org/installer | Phpchmod a+x composer.pharalias composer= '/usr/local/bin/composer.phar '

Installing Php-resque with composer

Suppose the web directory is in/opt/htdocs

Apt-get install git git-corecd/opt/htdocsgit clone Git://github.com/chrisboulton/php-resque.gitcd php-resquecomposer Install

Use of Php-resqueWrite a worker

In fact, Php-resque has given a simple example, the demo/job.php file is the simplest job:

Class php_job{Public Function perform () {sleep (120);    Fwrite (STDOUT, ' hello! '); }}

The job is to output characters to stdout after 120 seconds hello!

In Resque's design, a job must have a perform method, and the worker will run the method automatically.

Inserting a job into a queue

Php-resque also gives the simplest insert queue implementation demo/queue.php:

if (Empty ($argv [1])) {die (' Specify the name of a job to add e.g, PHP queue.php php_job ');} Require __dir__. '/init.php ';d ate_default_timezone_set (' GMT '); Resque::setbackend (' 127.0.0.1:6379 '); $args = Array (' time ' = = Time (), ' array ' = = Array (' test ' = = ' t Est ',),); $jobId = Resque::enqueue (' Default ', $ARGV [1], $args, true); echo "Queued job". $jobId. " \ n ";

In this example, queue.php needs to run as a CLI, insert the first parameter received by the CLI as the job name, plug in a queue named ' Default ', and output the job Id of the queue just inserted into the screen. In Terminal input:

PHP demo/queue.php Php_job

Results can be seen on-screen output:

Queued Job B1f01038e5e833d24b46271a0e31f6d6

That is, the job has been added successfully. Note that the job name here is consistent with the job class name we wrote: Php_job

To view job run status

Php-resque also provides an example of viewing the status of a job running directly:

PHP demo/check_status.php B1f01038e5e833d24b46271a0e31f6d6

You can see the output as:

Tracking status of B1f01038e5e833d24b46271a0e31f6d6. Press [break] to stop. Status of B1f01038e5e833d24b46271a0e31f6d6 Is:1

The job status we just created is 1. In Resque, a job has the following 4 statuses:

• resque_job_status::status_waiting = 1; Wait

• resque_job_status::status_running = 2; (executing)

• resque_job_status::status_failed = 3; Failed

• Resque_job_status::status_complete = 4; End

Because no worker is running, the job you just created is still waiting.

Running worker

This time we write demo/resque.php directly:

<?php date_default_timezone_set (' GMT ');    Require ' job.php '; Require '. /bin/resque ';

You can see that a worker needs at least two parts:

1. You can include the job class file directly, or you can use PHP's auto-load mechanism to specify the path of the job class and enable automatic loading.

2. Contains the default worker:bin/resque for Resque

Run in Terminal:

Queue=default PHP demo/resque.php

The previous queue section is the set environment variable, and we specify that the current worker is only responsible for handling the default queue. You can also use

queue=* PHP demo/resque.php

To process all queues.

After running the output is

#!/usr/bin/env php*** starting worker

Check with the PS command:

PS aux | grep resque

You can see that a PHP daemon is already running.

4607 0.0 0.1 74816 11612 pts/3 s+ 14:52 0:00 php demo/resque.php

Before using the Check Job command

PHP demo/check_status.php B1f01038e5e833d24b46271a0e31f6d6

You can see it in 2 minutes.

Status of B1f01038e5e833d24b46271a0e31f6d6 Is:4

The task is complete and the output hello! should be visible on the screen

At this point we have successfully completed a full demonstration of one of the simplest resque instances, and more complex scenarios and legacy issues are described in the next log.

Using PHP to implement daemon tasks background running and multithreading (Php-resque instructions)