This article mainly introduces the content is about the laravel of the queue system Introduction, has a certain reference value, now share to everyone, the need for friends can refer to
The Laravel queue provides a unified API for different background queue services, such as Beanstalk,amazon SQS, Redis, and even other relational database-based queues. The goal of the queue is to delay processing time-consuming tasks, such as sending messages, to drastically shorten Web requests and the corresponding time.
The queue configuration file is stored in the config/queue.php
. Each queue-driven configuration can be found in the file, including databases, BEANSTALKD, Amazon SQS, Redis, and synchronous (local-use) drivers. It also contains a null
queue driver for those tasks that abandon the queue.
Connect Vs. Queue
It is important to understand the difference between "connection" and "queue" before starting to use the Laravel queue. In your config/queue.php
configuration file, there is a connections
configuration option. This option defines a unique connection for Amazon SQS, Beanstalk, or a back-end service such as Redis. Either way, a given connection may have multiple "queues", while a "queue" can be considered to be a different stack or a large number of queue tasks.
Note that queue
a property is included in the configuration example for each connection in the configuration file queue
. This is the default queue, which is distributed to the queue when the task is sent to the specified connection. In other words, if you do not explicitly define a queue when you distribute the task, it is placed queue
in the queue defined by the properties in the connection configuration:
This task will be distributed to the default queue ... dispatch (new job);//This task will be sent to the "emails" queue ... dispatch ((New Job)->onqueue (' emails '));
Some applications may not need to send tasks to different queues, but only to a simple queue. However, it is still very useful to push tasks to different queues because the Laravel queue processor allows you to define the priority of the queue, so you can prioritize different queues or distinguish different ways of handling different tasks. For example, if you push the task into the high
queue, you can have the queue processor prioritize the tasks:
PHP Artisan queue:work--queue=high,default
Required settings for the drive
Database
To use database
This queue driver, you need to create a data table to store the task, and you can use queue:table
this Artisan command to create a migration of this data table. Once the migration is created, you can use migrate
this command to create a data table:
PHP Artisan queue:tablephp Artisan Migrate
Redis
In order to use the redis
queue driver, you need to config/database.php
configure the Redis database connection in your configuration file
If your Redis queue connection is using a Redis cluster, your queue name must contain a key hash tag. This is to ensure that all redis keys are placed in the same hash for a given queue:
' Redis ' = [ ' driver ' = ' redis ', ' connection ' = ' default ', ' queue ' = ' = ' {default} ', ' Retry_after ' = 90,],
Other queue-driven dependency expansion packs
Before you use the queue service in the list, you must install the following dependent expansion packs:
Create a task
Build Task Class
In your application, the queue's task class is placed by default in the app/Jobs
directory, and if the directory does not exist, the directory will be created automatically when you run the make:job
artisan command. You can use the following Artisan command to generate a new queue task:
PHP Artisan make:job Sendreminderemail
The generated class implements the Illuminate\Contracts\Queue\ShouldQueue
interface, which means that the task will be pushed into the queue instead of being executed synchronously.
Task class Structure
The structure of a task class is simple and generally contains only one method that the queue uses to invoke this task handle
. Let's look at an example of the task class, in this example, let's say we manage a podcast publishing service that needs to process the propagated guest files before publishing:
<?phpnamespace app\jobs;use app\podcast;use App\AudioProcessor;use Illuminate\bus\queueable;use Illuminate\queue\serializesmodels;use Illuminate\queue\interactswithqueue;use Illuminate\contracts\queue\shouldqueue;class Processpodcast implements shouldqueue{use Interactswithqueue, Queueable, Serializesmodels; protected $podcast; /** * Create a new task instance. * * @param podcast $podcast * @return void */Public Function __construct (podcast $podcast) {$ This->podcast = $podcast; }/** * Run the task. * * @param audioprocessor $processor * @return void */Public function handle (audioprocessor $processor) {//Process uploaded podcast ...}}
Note that in this example, we pass a eloquent model directly in the constructor of the task class. Because we quoted this in the task class. SerializesModels
, so that the eloquent model can be gracefully serialized and deserialized when it comes to processing tasks. If your queue task class receives a eloquent model in the constructor, only the attributes that recognize the model are serialized into the queue. When the task is actually running, the queue system will automatically retrieve the complete model from the database. This whole process is completely transparent to your application, which avoids some of the problems of serializing a complete eloquent pattern instance.
The method is called when the queue processes the task, handle
and here we can also let the Laravel service container automatically inject the dependent object through the handle method's parameter type hint.
{Note} This binary data, like picture content, must be converted using a method before it is put into a queue task base64_encode
. Otherwise, when this task is placed in the queue, it may not be serialized correctly as JSON.
Distributing tasks
Once you've written the task class, you'll be able dispatch
to distribute it through auxiliary functions. The only argument that needs to dispatch
be passed is an instance of this task class:
<?phpnamespace app\http\controllers;use app\jobs\processpodcast;use illuminate\http\request;use App\Http\ Controllers\controller;class Podcastcontroller extends controller{ /** * Save podcasts. * * @param request $request * @return Response * /Public function store (request $ Request) { //create podcast ... Dispatch (New Processpodcast ($podcast));} }
{Tip} dispatch
Provides a simple, globally available function that is also very easy to test. Check out the Laravel test documentation below to learn more.
Deferred distribution
If you want to delay the execution of a task in a queue, you can use the method of the task instance delay
. This method is Illuminate\Bus\Queueable
provided by trait, and this trait is loaded by default in all auto-generated task classes. For deferred tasks we can give an example, for example, to specify a task to be performed after 10 minutes of distribution:
<?phpnamespace app\http\controllers;use carbon\carbon;use app\jobs\processpodcast;use Illuminate\Http\Request; Use App\http\controllers\controller;class Podcastcontroller extends controller{ /** * Save a new podcast. * * @param request $request * @return Response * /Public function store (request $ Request) { //create podcast ... $job = (new Processpodcast ($podcast)) ->delay (Carbon::now ()->addminutes ()); Dispatch ($job); }}
{Note} The Amazon SQS Queue service has a maximum delay of 15 minutes.
customizing Queues & Connections
Distributing tasks to a specified queue
By pushing tasks to different queues, you can categorize queue tasks and even control how many tasks are assigned to different queues. Remember, this is not about pushing tasks into different "connections" in the queue configuration file, but instead pushing them to a different queue in a connection. To specify a queue, you invoke the method of the task instance onQueue
:
<?phpnamespace app\http\controllers;use app\jobs\processpodcast;use illuminate\http\request;use App\Http\ Controllers\controller;class Podcastcontroller extends controller{ /** * Save a new podcast. * * @param request $request * @return Response * /Public function store (request $ Request) { //create podcast ... $job = (new Processpodcast ($podcast))->onqueue (' processing '); Dispatch ($job); }}
Distributing tasks to a specified connection
If you use multiple queue connections, you can push the task to the specified connection. To specify a connection, you can invoke the method of the task instance onConnection
:
<?phpnamespace app\http\controllers;use app\jobs\processpodcast;use illuminate\http\request;use App\Http\ Controllers\controller;class Podcastcontroller extends controller{ /** * Save a new podcast. * * @param request $request * @return Response * /Public function store (request $ Request) { //create podcast ... $job = (new Processpodcast ($podcast))->onconnection (' Sqs '); Dispatch ($job); }}
Of course, you can chain-call onConnection
and onQueue
to specify both the connection and the queue for the task:
$job = (new Processpodcast ($podcast)) ->onconnection (' Sqs ') ->onqueue (' processing ');
Specify the maximum number of retries/timeout values for a task
Maximum number of attempts
Specify the maximum number of attempts in a task that you can try to set by using the Artisan command line --tries
:
PHP Artisan queue:work--tries=3
However, you can take a more refined approach to doing this, such as defining the maximum number of attempts in a task class. If the maximum number of attempts is defined in both the class and the command line, Laravel will take precedence over the values in the task class:
<?phpnamespace App\jobs;class Processpodcast implements shouldqueue{ /** * Task Max Retries * * @var int */Public $tries = 5;}
Timeout
Similarly, the maximum number of seconds a task can run can be specified using the switch on the Artisan command line --timeout
:
PHP Artisan queue:work--timeout=30
However, you can also define a variable in the task class to set the maximum description that can be run, and if the maximum number of attempts is defined on both the class and the command line, Laravel will take precedence over the values in the task class:
<?phpnamespace App\jobs;class Processpodcast implements shouldqueue{ /** * Time-out for tasks to run. * * @var int */public $timeout = 120;}
Error handling
If an exception is thrown when the task runs, the task is automatically released back into the queue so that it can be re-run again. If you continue to throw an exception, the task will continue to be freed back to the queue until the number of retries reaches the maximum number of times your app allows. The maximum number of times is queue:work
defined by parameters when invoking the Artisan command --tries
. More information on the queue processor can be seen below.
Running the queue processor
The Laravel contains a queue processor that can handle these tasks when a new task is pushed into the queue. You can run the processor with the Queue:work Artisan command. Note that once the queue:work
command starts, it will run until you manually stop or you close the console:
PHP Artisan Queue:work
{tip} to keep queue:work
the process running in the background permanently, you should use process monitoring tools, such as Supervisor
to ensure that the queue processor does not stop running.
Be sure to remember that the queue processor is a long-running process and holds the application state that has started in memory. The result is that if you modify the code after the processor is running, those changes will not be applied to the processor. So in your redeployment process, be sure to restart the queue processor.
Specify Connections & Queues
You can specify the connection that the queue processor uses. You config/queue.php
define multiple connections in a configuration file, and work
the connection name you pass to the command is at least consistent with one of them:
PHP Artisan queue:work Redis
You can customize the queue processor by processing a specific queue for a given connection. For example, if all your messages are processed in a redis
queue in the connection emails
, you can start a queue processor that only processes that particular queue with the following command:
PHP Artisan queue:work Redis--queue=emails
Resource considerations
The daemon queue does not "restart" the framework before each job is processed. Therefore, after each task is completed, you should release any resources that are too large to occupy. For example, if you use the GD library for image processing, you should use the imagedestroy
free memory after completion.
Queue priority
Sometimes you want to set the priority of the processing queue. For example config/queue.php
, you might have set the redis
default queue priority in the connection low
, but you might occasionally want to push a task into the high
priority queue, like this:
Dispatch (New Job)->onqueue (' High ');
To verify that the tasks in the high
queue are low
processed before the tasks in the queue, you start a queue processor, pass the list of queue names to it and comma, interval:
PHP Artisan queue:work--queue=high,low
Queue Processor & Deployment
Because the queue processor is a long-lived process, if the code changes and the queue processor does not restart, they cannot apply the new code. So the simplest way is to restart the queue processor during the redeployment process. You can only enter it gracefully queue:restart
to restart all queue processors.
PHP Artisan Queue:restart
This command will tell all queue processors to end the process after the current task has been completed so that no task is lost. Because the queue processor is executing the queue:restart
command against the end process, you should run a process manager, such as Supervisor from the restart queue processor.
Task Expiration & Timeout
Task expires
config/queue.php
In the configuration file, each queue connection defines an retry_after
option. This option specifies how many seconds the task will be processed and is retried as a failure. For example, if this option is set to 90
, then it will be released back to the queue when the task continues to execute for a 90
second without being deleted. Typically, you should retry_after
set the time that corresponds to the longest-consuming task.
{Note} The only connection that does not have an retry_after
option is Amazon SQS. When using Amazon SQS, you must configure this retry threshold through the Amazon command line.
Queue Processor Timeout
queue:work
The Artisan command has an external --timeout
option. This option specifies Laravel
how long the queue processor should be shut down after the maximum execution time. Sometimes a queue's subprocess is zombie for a number of reasons, such as an external HTTP request that is not responding. This --timeout
option removes the zombie process that exceeds the specified event limit.
PHP Artisan queue:work--timeout=60
retry_after
Configuration options and --timeout
command-line options are different, but you can work at the same time to ensure that tasks are not lost and do not repeat.
{Note} --timeout
Should always be retry_after
at least a few seconds longer than the short. This ensures that the task process can always be killed before the failure is retried. If your --timeout
option is greater than the retry_after
configuration option, your task may be executed two times.
Queue Process Sleep Time
When a queue needs to process a task, the process continues to process the task with no delay between them. However, if no new work is available, the sleep
parameters determine how long the worker process will "sleep":
PHP Artisan queue:work--sleep=3
Supervisor Configuration
Installing Supervisor
Supervisor is a process monitoring software on a Linux operating system that automatically restarts queue:listen
queue:work
after or after a command failure. To install Supervisor in Ubuntu, you can use the following command:
sudo apt-get install Supervisor
{Tip} If you manually configure Supervisor to sound a little difficult to cope with, consider using Laravel Forge, which can automatically install and configure Supervisor for your Laravel project.
Configure Supervisor
Supervisor profiles are typically placed in a /etc/supervisor/conf.d
directory where you can create any number of profiles that require Supervisor to monitor your process. For example we create one laravel-worker.conf
to start and monitor a queue:work
process:
[program:laravel-worker]process_name=% (Program_name) s_% (process_num) 02dcommand=php/home/forge/app.com/artisan Queue:work SQS--sleep=3--tries=3autostart=trueautorestart=trueuser=forgenumprocs=8redirect_stderr=truestdout_ Logfile=/home/forge/app.com/worker.log
The commands in this example numprocs
require Supervisor to run and monitor 8 queue:work
processes, and restart after they fail to run. Of course, you have to change the command
command queue:work sqs
to show the queue driver of your choice.
Start Supervisor
When this configuration file is created, you need to update the Supervisor configuration and start the process with the following command:
sudo supervisorctl rereadsudo supervisorctl updatesudo supervisorctl start laravel-worker:*
For more information on setting up and using Supervisor, please refer to the Supervisor official documentation.
Handle a failed task
Sometimes the tasks in your queue will fail. Don't worry, things will not go smoothly. The Laravel has a convenient way to specify the maximum number of times a task retries. When a task exceeds this retry count, it is inserted into the failed_jobs
data table. To create failed_jobs
a table, you can use the queue:failed-table
command:
PHP Artisan queue:failed-tablephp Artisan Migrate
Then run the queue processor, and you should --tries
specify the maximum number of retries for the task by parameter when calling the Queue:work command. If not specified, the task will be retried permanently:
PHP Artisan queue:work Redis--tries=3
Purge failed tasks
You can define the method directly in the task class failed
, which can run the purge logic of the task when the task fails. This place is great for sending a warning to a user or resetting a task to perform. The exception information that causes the task to fail is passed to the failed
method:
<?phpnamespace app\jobs;use exception;use app\podcast;use app\audioprocessor;use Illuminate\Bus\Queueable;use Illuminate\queue\serializesmodels;use Illuminate\queue\interactswithqueue;use Illuminate\Contracts\Queue\ Shouldqueue;class Processpodcast implements shouldqueue{use interactswithqueue, queueable, serializesmodels; protected $podcast; /** * Create a new task instance. * * @param podcast $podcast * @return void * /Public Function __construct (podcast $ Podcast) { $this->podcast = $podcast; } /** * Perform tasks. * * @param audioprocessor $processor * @return void * /Public Function handle ( Audioprocessor $processor) { //processing on the spread of the guest ... } /** * Failed task to process. * * @param Exception $exception * @return void * /Public Function failed ( Exception $exception) { //Send failure notification to user, etc.. }}
Task Failure Events
If you want to register an event that will be called when a queue task fails, you can use the Queue::failing
method. This will give you the opportunity to notify your team by e-mail or hipchat through this event. For example, we can AppServiceProvider
attach a callback function to this event in the Laravel built-in:
<?phpnamespace App\providers;use Illuminate\support\facades\queue;use Illuminate\queue\events\jobfailed;use Illuminate\support\serviceprovider;class Appserviceprovider extends serviceprovider{ /** * Start Services for any application. * * @return void */public function boot () { queue::failing (jobfailed $event) { //$event->connectionname //$event->job //$event->exception }); } /** * Registered service provider. * * @return void * /Public Function register () { // }}
Retry failed Task
To see failed_jobs
all of your failed tasks in the datasheet, you can use queue:failed
this Artisan command:
PHP Artisan queue:failed
queue:failed
The command lists the ID, connection, queue, and failure time of all tasks, and the task ID can be used to retry the failed task. For example, to retry a 5
failed task with an ID, the command is as follows:
PHP Artisan queue:retry 5
To retry all failed tasks, you can use queue:retry
and use all
as ID:
PHP Artisan Queue:retry All
If you want to remove a failed task, you can use the queue:forget
command:
PHP Artisan queue:forget 5
queue:flush
The command allows you to remove all failed tasks:
PHP Artisan Queue:flush
Task events
Using the queue before
and after
methods, you can specify the callback processing before and after the task is processed. In these callbacks, it is a good time to implement additional logging or to increase the statistical data. Typically, you should call these methods in the service container. For example, we use the Appserviceprovider in Laravel:
<?phpnamespace App\providers;use Illuminate\support\facades\queue;use Illuminate\support\serviceprovider;use Illuminate\queue\events\jobprocessed;use Illuminate\queue\events\jobprocessing;class AppServiceProvider extends serviceprovider{ /** * Start any service. * * @return void */public function boot () { queue::before (jobprocessing $event) { //$event->connectionname //$event->job //$event->job->payload () }); Queue::after (function (jobprocessed $event) { //$event->connectionname //$event->job //$ Event->job->payload () }); } /** * Registered service provider. * * @return void * /Public Function register () { // }}
队列
using the method in facade looping
, you can try to execute the specified callback method before the queue gets the task. For example, you can use a closure to roll back a transaction that has failed a task before.
queue::looping (function () {while (Db::transactionlevel () > 0) {db::r Ollback (); }});