Use a Supervisor to manage your Laravel queue

Use a Supervisor to manage your Laravel queue

The Laravel official website does not mention using it to write CLI applications, that is, daemon processes or executable scripts. However, it provides more convenient Queue functions.

Laravel queue

During application development, we will inevitably encounter the need to process time-consuming tasks. If these tasks are processed directly in user requests, the page display will be blocked. Although some features of fastcgi can be used to output the page and execute background tasks, it is much easier to hand over tasks to asynchronous queues for processing.

Configure and start

The Laravel queue function provides us with a convenient way to process these asynchronous tasks. configuring a queue only takes the following steps:

1. Configurationapp/config/queue.phpIndefaultThe configuration item is the queue system in the system,syncIs a direct execution, not an asynchronous queue.
2. Create a queue processing class, as shown in figureSendMail. For the class file location, refer to the other article using your own class library in Laravel.
3. Push a task in the application to the queueQueue::push('SendMail')
4. Start Laravel queue listenerphp artisan queue:listenOr usephp artisan queue:workProcess a message in the queue Header

Laravel queue Parallel Processing

If you have used the Laravel queue,queue:listenLinear execution means that the next task is read only after a task is completed. This does not meet our daily asynchronous time-consuming task processing needs. Therefore, we recommend that you start multiplequeue:listen.

php artisan queue:listen && php artisan queue:listen ...

This is theoretically feasible because the program does not conflict with each other with the help of asynchronous queues. However, due to the memory processing defects of PHP, it is difficult to ensure that a program running in the background for a long time does not leak memory, suchqueue:listenSuch an endless loop program. Therefore, in the formal environment, we prefer to use multiplequeue:workExecute tasks in the asynchronous queue in parallel.queue:workIt only reads the first task of the team. After the execution is complete, the program ends. If there is no task, the program ends. This method is similar to PHP's processing of WEB requests without memory leakage.

With Supervisor, you can easily createqueue:workAsynchronous queue parallel processing.

Supervisor

Supervisor is a process control system written in python. It provides a large number of functions to manage processes.

1. Multi-process startup of the program. You can configure the number of processes started at the same time without the need to start them one by one.
2. The exit code of the program. You can determine whether automatic restart is required based on the exit code of the program.
3. Processing of logs generated by the program
4. Environment for process initialization, including directories, users, umask, and signals required to shut down processes, etc.
5. Web interface for manual process management (start, start, restart, view Process status), and xmlrpc Interface

Install

pip install supervisor

Configuration

The configuration item example is as follows. A unique Laravel configuration will be created later.

; Sample supervisor config file.; For more information on the config file, please see :; http://supervisord.org/configuration.html ; Note: shell expansion ("~ "Or" $ HOME ") is not supported. environment; variables can be expanded using this syntax: "% (ENV_HOME) s ". [unix_http_server]; file =/tmp/supervisor for supervisord unix socket service configuration. sock; directory for saving socket files; chmod = 0700; socket File Permission (default 0700); chown = nobody: nogroup; socket owner and group name; username = user; by default, you do not need to log on to the user (open server); password = 123; by default, you do not need to log on to the password (open server); [inet_http_server]; supervisord tcp service configuration; port = 127.0.0.1: 9001; tcp port; username = user; tcp login user; password = 123; tcp login password [supervisord]; Master Process configuration of supervisord logfile =/tmp/supervisord. log; main process log configuration logfile_maxbytes = 50 MB; maximum log volume, 50 mblogfile_backups = 10 by default; number of log File backups, 10 loglevel = info by default; log Level, default info; and: debug, warn, tracepidfile =/tmp/supervisord. pid; supervisord's pidfile file nodaemon = false; whether to start minfds = 1024 as a daemon; minimum valid file descriptor, 1024 minprocs = 200 by default; minimum valid process descriptor, 200 by default; umask = 022; umask of the Process file; default value: 200; user = chrism; default value: current user; required if root; identifier = supervisor; expression of supervisord, default Value: 'supervisor '; directory =/tmp; default value: No cd to the current directory; nocleanup = true; default value: false; childlogdir =/tmp; ('auto' child log dir, default $ TEMP); environment = KEY = value; the initial KEY-value pair is passed to the process; strip_ansi = false; (strip ansi escape codes in logs; def. false); the below section must remain in the config file for RPC; (supervisorctl/web interface) to work, additional interfaces may be; added by defining them in separate rpcinterface: sections [rpcinterface: supervisor] supervisor. rpcinterface_factory = supervisor. rpcinterface: make_main_rpcinterface [supervisorctl] serverurl = unix: // tmp/supervisor. sock; use a unix: // URL for a unix socket; serverurl = http://127.0.0.1:9001 ; Use an http: // url to specify an inet socket; username = chris; if set, it should be the same as http_username; password = 123; if set, it should be the same as http_password; prompt = mysupervisor; command Line prompt, default "supervisor"; history_file = ~ /. SC _history; command line history; The below sample program section shows all possible program subsection values,; create one or more 'real' program: sections to be able to control them under; supervisor .; [program: theprogramname]; command =/bin/cat; running program (you can use the parameter relative to the PATH); process_name = % (program_name) s; process name expression, the default value is % (program_name) s; numprocs = 1; the default number of processes started, the default value is 1; directory =/tmp; cwd to the specified directory before running, cmd is not executed by default; umask = 022; umask process; None by default; priority = 999; priority of program running, 999 by default; autostart = true; automatically started with supervisord by default, true by default; autorestart = unexpected; whether/when to restart (default: unexpected); startsecs = 1; number of secs prog must stay running (def. 1); startretries = 3; max # of serial start failures (default 3); exitcodes =; expected exit code, default:; stopsignal = QUIT; process killing signal, default TERM; stopwaitsecs = 10; max num secs to wait b4 SIGKILL (default 10); stopasgroup = false; send a stop signal to a unix process group; default value: false; killasgroup = false; send a SIGKILL signal to the unix process group. The default value is false. user = chrism; setuid is set for the unix account that runs the program; redirect_stderr = true; standard error is redirected to standard output. The default value is false; stdout_logfile =/a/path; standard output file path NONE = none; default AUTO; stdout_logfile_maxbytes = 1 MB; max # logfile bytes b4 rotation (default 50 MB); stdout_logfile_backups = 10; # of stdout logfile backups (default 10); stdout_capture_maxbytes = 1 MB; number of bytes in 'capturemode' (default 0); stdout_events_enabled = false; emit events on stdout writes (default false); stderr_logfile =/a/path; stderr log path, NONE for none; default AUTO; stderr_logfile_maxbytes = 1 MB; max # logfile bytes b4 rotation (default 50 MB); stderr_logfile_backups = 10; # of stderr logfile backups (default 10); stderr_capture_maxbytes = 1 MB; number of bytes in 'capturemode' (default 0); stderr_events_enabled = false; emit events on stderr writes (default false); environment = A = 1, B = 2; process environment additions (def no adds); serverurl = AUTO; override serverurl computation (childutils); The below sample eventlistener section shows all possible; eventlistener subsection values, create one or more 'real'; eventlistener: sections to be able to handle event notifications; sent by supervisor .; [eventlistener: theeventlistenername]; command =/bin/eventlistener; running program (the parameter can be used relative to the PATH); process_name = % (program_name) s; process name expression, the default value is % (program_name) s; numprocs = 1; The default value is 1; events = EVENT; event notif. types to subscribe to (req 'd); buffer_size = 10; event Buffer Queue size, 10 by default; directory =/tmp; cwd to the specified directory before running, by default, cmd is not executed; umask = 022; umask process; default value: None; priority =-1; priority of program running; default value:-1; autostart = true; automatically started with supervisord by default, default Value: true; autorestart = unexpected; whether/when to restart (default: unexpected); startsecs = 1; number of secs prog must stay running (def. 1); startretries = 3; max # of serial start failures (default 3); exitcodes =; expected exit code, default:; stopsignal = QUIT; process killing signal, default TERM; stopwaitsecs = 10; max num secs to wait b4 SIGKILL (default 10); stopasgroup = false; send a stop signal to a unix process group; default value: false; killasgroup = false; send SIGKILL signals to unix process groups. the default value is false. user = chrism; setuid to this UNIX account to run the program; redirect_stderr = true; redirect proc stderr to stdout (default false ); stdout_logfile =/a/path; stdout log path, NONE for none; default AUTO; stdout_logfile_maxbytes = 1 MB; max # logfile bytes b4 rotation (default 50 MB); stdout_logfile_backups = 10; # of stdout logfile backups (default 10); stdout_events_enabled = false; emit events on stdout writes (default false); stderr_logfile =/a/path; stderr log path, NONE for none; default AUTO; stderr_logfile_maxbytes = 1 MB; max # logfile bytes b4 rotation (default 50 MB); stderr_logfile_backups; # of stderr logfile backups (default 10); stderr_events_enabled = false; emit events on stderr writes (default false); environment = A = 1, B = 2; process environment additions; serverurl = AUTO; override serverurl computation (childutils ); the below sample group section shows all possible group values,; create one or more 'real' group: sections to create "heterogeneous"; process groups .; [group: thegroupname]; programs = progname1, progname2; Any x defined in [program: x]; priority = 999; program running priority, 999 by default; the [include] section can just contain the "files" setting. this; setting can list multiple files (separated by whitespace or; newlines ). it can also contain wildcards. the filenames are; interpreted as relative to this file. included files * cannot *; include files themselves .; [include]; files = relative/directory /*. ini

Laravel Configuration

In the include of the supervisor, we can createSendMailProject

[program:waaQueue]command                 = php artisan queue:workdirectory               = /path/to/appprocess_name            = %(program_name)s_%(process_num)snumprocs                = 6autostart               = trueautorestart             = truestdout_logfile          = /path/to/app/storage/logs/supervisor_waaQueue.logstdout_logfile_maxbytes = 10MBstderr_logfile          = /path/to/app/storage/logs/supervisor_wqqQueue.logstderr_logfile_maxbytes = 10MB

Start

1. Start supervisord and executesupervisordYou can find the configuration file in the default directory.
2. Runsupervisorctl helpTo view the available commands

This article permanently updates the link address: