Analysis of Swoole and Swoft of Swoft source code

This article to share the content is about Swoft source analysis of the swoole and swoft some of the introduction (Task delivery/timing Task), there is a certain reference value, the need for friends can refer to.

Objective

SwoftThe task function is based on Swoole Task机制 , or Swoft the Task nature of the mechanism is the Swoole Task机制 encapsulation and reinforcement of the right.

Task delivery

Swoft\task\task.phpclass task{/** * Deliver coroutine or Async Task * * @param string $taskName * @p Aram String $methodName * @param array $params * @param string $type * @param int $timeout * * @re  Turn Bool|array * @throws taskexception */public static function deliver (string $taskName, String $methodName, Array $params = [], string $type = Self::type_co, $timeout = 3) {$data = Taskhelper::p ack ($taskName, $method        Name, $params, $type); if (! App::isworkerstatus () &&! App::iscocontext ()) {return self::d eliverbyqueue ($DATA);//See command section below} if (!         App::isworkerstatus () && app::iscocontext ()) {throw new Taskexception (' Please deliver task by http! ');        } $server = App:: $server->getserver ();            Delier coroutine Task if ($type = = Self::type_co) {$tasks [0] = $data; $prifleKey = ' Task '. '.' . $taskName. '.' . $methodName;            APP::p Rofilestart ($prifleKey);            $result = $server->taskco ($tasks, $timeout);            APP::p rofileend ($prifleKey);        return $result;    }//Deliver async task return $server->task ($data); }}

The task post Task::deliver() packages The calling parameter after it $type has been passed by the parameter Swoole $server->taskCo() or the interface is $server->task() posted to Task进程 .
Taskitself is always synchronous execution, $type only affect the behavior of the delivery of this operation, Task::TYPE_ASYNC corresponding to the $server->task() asynchronous delivery, Task::deliver() return immediately after the call, Task::TYPE_CO the corresponding $server->taskCo() is the process of delivery, after delivery of the process Control, task completion or execution timeout Task::deliver() To be returned from the co-process.

Task execution

Swoft\task\bootstrap\listeners\taskeventlistener/** * The listener of the Swoole Task * @SwooleListener ({* Swooleevent     :: On_task, * swooleevent::on_finish, *}) */class Taskeventlistener implements Taskinterface, finishinterface{/**  * @param \swoole\server $server * @param int $taskId * @param int $workerId * @param Mixed $data * @return Mixed * @throws \invalidargumentexception */Public function Ontask (Server $server, int $taskId, int $workerId, $data) {try {/* @var taskexecutor $taskExecutor */$            Taskexecutor = App::getbean (Taskexecutor::class);        $result = $taskExecutor->run ($data); } catch (\throwable $throwable) {app::error (sprintf (' taskexecutor->run%s file=%s line=%d ', $throwable-&gt            ; GetMessage (), $throwable->getfile (), $throwable->getline ()));            $result = false; Release system Resources App::trigger (Appevent::resource_release);        App::trigger (Taskevent::after_task);    } return $result; }}

Here is swoole.onTask the event callback, whose responsibility is simply Worker to forward the packaged data sent by the process TaskExecutor .

SwooleThe Task nature of the mechanism is to Worker进程 post time-consuming tasks to synchronous Task进程 (aka TaskWorker ) processing, so swoole.onTask that the event callbacks are Task进程 executed in. As mentioned above, Worker进程 it is the environment in which most of your HTTP service code executes, but starting with the TaskEventListener.onTask() method, the execution environment of the code is Task进程 , in other words, TaskExecutor and the concrete TaskBean is executed in Task进程 .

//swoft\task\taskexecutor/** * the Task executor * * @Bean () */class taskexecutor{ /** * @param string $data * @return Mixed */Public function run (string $data) {$data = Taskhel        Per::unpack ($data);        $name = $data [' name '];        $type = $data [' type '];        $method = $data [' method '];        $params = $data [' params ']; $logid = $data [' Logid ']??        Uniqid (', true); $spanid = $data [' Spanid ']??        0;        $collector = Taskcollector::getcollector ();        if (!isset ($collector [' Task '] [$name])) {return false;        } list (, $coroutine) = $collector [' Task '] [$name];        $task = App::getbean ($name);        if ($coroutine) {$result = $this->runcotask ($task, $method, $params, $logid, $spanid, $name, $type);        } else {$result = $this->runsynctask ($task, $method, $params, $logid, $spanid, $name, $type);    } return $result; }}

Task execution idea is very simple, will Worker进程 send the data unpacked into the original call parameters, according to the $name parameters to find corresponding TaskBean and call its corresponding task() method. where TaskBean class-level annotations @Task(name="TaskName") or @Task("TaskName") declarations are used.

It is worth mentioning that, @Task in addition to name attributes, annotations also have a coroutine property, the above code will select the use of the process runCoTask() or synchronous execution according to this parameter runSyncTask() Task . However, due to the fact that the Swoole Task进程 execution is fully synchronized and does not support the co-process, the current version does not configure the parameter to be true . The same TaskBean task code that is written in must be either synchronous blocking or the ability to automatically downgrade asynchronous nonblocking and co-processes to synchronous blocking based on the environment

Post a task from the process

Earlier we mentioned:

Swoole The Task nature of the mechanism is to Worker进程 deliver time-consuming tasks to synchronous Task进程 (aka TaskWorker ) processing.

In other words, swoole 's $server->taskco () or $server->task () are available only in the Used in the worker process .
This restriction greatly limits the use of scenarios. How can I be able to post a task in Process ? Swoft provides a Task::d eliverbyprocess () method to circumvent this restriction. Its implementation principle is also simple, through the $server->sendmessage () method of Swoole to post the invocation information from Process to Worker process , which is then posted by the worker process to the task process , the relevant code is as follows:

swoft\task\task.php/** * Deliver Task by Process * * @param string $taskName * @param string $methodName * @param array< c0/> $params * @param string $type * @param int    $timeout * @param int    $workId * * @return bool */public static func tion deliverbyprocess (String $taskName, String $methodName, array $params = [], int $timeout = 3, int $workId = 0, string $type = Self::type_async): bool{/    * @var pipemessageinterface $pipeMessage */    $server      = App:: $server Getserver ();    $pipeMessage = App::getbean (pipemessage::class);    $data = [        ' name ' = =    $taskName,        ' method ' =  $methodName,        ' params ' =  $params,        ' timeout ' = $timeout,        ' type '    = ' $type,    ];    $message = $pipeMessage->pack (pipemessage::message_type_task, $data);    return $server->sendmessage ($message, $workId);}

After the data is packaged, use the $server->sendMessage() post to Worker :

swoft\bootstrap\server\servertrait.php/** * Onpipemessage Event Callback * * @param \swoole\server $server * @param int< c0/> $srcWorkerId * @param string         $message * @return void * @throws \invalidargumentexception */public function Onpipem Essage (Server $server, int $srcWorkerId, string $message) {/    * @var pipemessageinterface $pipeMessage */    $ Pipemessage = App::getbean (pipemessage::class);    List ($type, $data) = $pipeMessage->unpack ($message);    App::trigger (appevent::P ipe_message, NULL, $type, $data, $srcWorkerId);}

$server->sendMessage, Worker进程 a callback for an event is triggered when the data is received swoole.pipeMessage , Swoft and it is converted to its own swoft.pipeMessage event and triggered.

//swoft\task\event\listeners\pipemessagelistener.php/** * the pipe message      Listener * * @Listener (event=appevent::P ipe_message) */class Pipemessagelistener implements eventhandlerinterface{/** * @param \swoft\event\eventinterface $event */Public Function handle (Eventinterface $event) {$para        ms = $event->getparams ();        if (count ($params) < 3) {return;        } list ($type, $data, $srcWorkerId) = $params;        if ($type! = pipemessage::message_type_task) {return;        } $type = $data [' type '];        $taskName = $data [' name '];        $params = $data [' params '];        $timeout = $data [' timeout '];        $methodName = $data [' method '];    Delever Task Task::d eliver ($taskName, $methodName, $params, $type, $timeout); }}

swoft.pipeMessageThe event is eventually PipeMessageListener processed. In the relevant listener, if the event is discovered, swoft.pipeMessage Task::deliverByProcess() Worker进程 It will be executed once and the Task::deliver() task data will be delivered to the end TaskWorker进程 .

A simple retrospective exercise: from Task::deliverByProcess() one TaskBean final execution to another, what processes have gone through, and which parts of the call chain are being executed in each?

To post a task from the command process or its child processes

swoft\task\queuetask.php/** * @param string $data * @param int    $taskWorkerId * @param int    $srcWorkerId * * @retu RN BOOL */public function deliver (string $data, int $taskWorkerId = NULL, $srcWorkerId = null) {    if ($taskWorkerId = = = NULL) {        $taskWorkerId = Mt_rand ($this->workernum + 1, $this->workernum + $this->tasknum);    }    if ($srcWorkerId = = = null) {        $srcWorkerId = Mt_rand (0, $this->workernum-1);    }    $this->check ();    $data   = $this->pack ($data, $srcWorkerId);    $result = \msg_send ($this->queueid, $taskWorkerId, $data, false);    if (! $result) {        return false;    }    return true;}

CommandThe situation can be a little more complicated for the process's task posting.
As mentioned above Process , it is often derived from Http/Rpc services, as Manager the descendants of the same process, they are able to get Swoole\Server the handle variables, thus through $server->sendMessage() , $server->task() and other methods for task delivery.

But in Swoft the system, there is a very pedestrian role: Command .
CommandProcesses are shell cronb Http/Rpc not related to service-dependent processes, starting from or independently. So Command processes and processes that Command start from within Process are not able to get Swoole\Server the call handle directly through the UnixSocket task post.
In order to provide task delivery support for this process, Swoft Swoole Task进程 A special feature is utilized---- Message Queuing .

The same project Command and Http\RpcServer through the contract one message_queue_key gets to the same message queue in the system kernel, and the Comand process can then pass the message queue to the Task进程 delivery task.
The mechanism does not provide an external public method, which is only included in the Task::deliver() method, and Swoft is implicitly switching the delivery mode based on the current environment. However, the implementation of this message queue relies on Semaphore expansion, and if you want to use it, you need to PHP add parameters at compile time --enable-sysvmsg .

Scheduled Tasks

In addition to the common tasks that are performed manually, a Swoft timer task function with a precision of seconds is provided to replace the Linux functionality in the project Crontab .

Swoft use two front process ---Task scheduling processes: crontimerprocess and task execution processes Cronexecprocess
, and two memory data tables----- runtimetable (Task (configuration) table) origintable (Task) execution table) is used for scheduled task management scheduling. The structure of each row of the
two table records is as follows:

\\swoft\task\crontab\tablecrontab.php/** * Task table, record user-configured task information * each row of the table contains the following fields, where ' rule ', ' taskclass ', ' Taskmethod ' Generate key uniquely determines a record * @var array $originStruct */private $originStruct = [' rule ' = = [\swoole\table::type_string, 1 00],//timed task execution rules, corresponding to @scheduled annotated cron properties ' taskclass ' = [\swoole\table::type_string, 255],//task name corresponding to @task's Name property (default to class name ' Taskmethod ' = [\swoole\table::type_string, 255],//task method, corresponding to @scheduled annotation method ' add_time ' = [\swoole\table] :: type_string, 11],//10-bit timestamp when initializing the contents of the table];/** * Execute the table, record the list of tasks to be performed in a short time and its execution status * table Each row of records contains the following fields, where ' taskclass ', ' taskmethod ', ' Minute ', ' sec ' generates key uniquely determines a record * @var array $runTimeStruct */private $runTimeStruct = [' taskclass ' = = [\swoole\table: : Type_string, 255],//ibid. ' Taskmethod ' = [\swoole\table::type_string, 255],//ibid. ' Minute ' = [\swoole\table] :: type_string, 20],//need to perform task time, accurate to minute format date (' Ymdhi ') ' sec ' = [\swoole\table::type_string, 20],//need to perform task time, exact To minutes 10 bit timestamp ' runstatus ' = [\swoole\table::typE_int, 4],//task status, has 0 (not executed) 1 (executed) 2 (in execution) of three. Note: The execution here is an easy to misunderstand place, the execution here does not mean the task itself execution, but the value of the "task delivery" operation of the execution, from the macro view of the undelivered _,_ has been delivered _,_ delivery _ description will be more accurate. ];

Why do I use Swoole's memory table here?

SwoftThe scheduled task management is the responsibility of the task scheduling process and the task execution process process respectively. Two processes run together to manage timed tasks, and array() two processes will inevitably require frequent interprocess communication if the use of an inter-process-independent structure is used. The use of cross-process Table (this article Table , unless specifically described, refers to Swoole the Swoole\Table structure) directly for inter-process data sharing, not only high performance, simple operation also decoupled two processes.

In order Table to be able to work together across two processes, Table you must Swoole Server create and allocate memory before booting. The specific code in Swoft\Task\Bootstrap\Listeners->onBeforeStart() , relatively simple, interested can read by themselves.

The background is over, let's take a look at the behavior of these two timed task processes

//swoft\task\bootstrap\process\crontimerprocess.php/** * Crontab Timer Process * * @Process (name= "Crontimer", Boot=true) */class crontimerprocess implements processinterface{/** * @para M \swoft\process\process $process */Public Function run (swoftprocess $process) {//code .../* @v        AR \swoft\task\crontab\crontab $cron */$cron = App::getbean (' Crontab ');        Swoole/httpserver $server = App:: $server->getserver ();        $time = (60-date (' s ')) * 1000; $server->after ($time, function () use ($server, $cron) {//Every minute check all tasks, and prepare the TA            SKS that next execution point needs $cron->checktask ();            $server->tick (), function () use ($cron) {$cron->checktask ();        });    }); }}

//swoft\task\crontab\crontab.php/** * Initialize runtimetable data * * @param array $task Task * @param the array $parseResult parse the result of the crontab command rule, that is, which seconds the task needs to execute in the current minute * @return bool */private function Initruntimetabledata    (array $task, array $parseResult): bool{$runTimeTableTasks = $this->getruntimetable ()->table;    $min = Date (' Ymdhi ');    $sec = strtotime (Date (' y-m-d h:i '));        foreach ($parseResult as $time) {$this->checktaskqueue (false);        $key = $this->getkey ($task [' rule '], $task [' Taskclass '], $task [' Taskmethod '], $min, $time + $sec); $runTimeTableTasks->set ($key, [' Taskclass ' + $task [' Taskclass '], ' taskmethod ' and ' = ' $task [' Taskmethod '], ' minute ' = $min, ' sec ' = + $time + $sec, ' runstatus ' => ;    Self::normal]); } return true;} 

The

crontimerprocess is the scheduled task scheduling process for Swoft , and its core method is Crontab->initruntimetabledata () .
The process uses the timer function of swoole , through swoole\timer callbacks executed at the first second of every minute, crontimerprocess Each wake-up will traverse the task table to calculate the list of tasks that need to be performed for 60 seconds in the current minute, write the execution table, and mark it as not executed.

//swoft\task\bootstrap\process/** * Crontab Process * * @Process (name= "cronexec     ", Boot=true) */class cronexecprocess implements processinterface{/** * @param \swoft\process\process $process        */Public Function run (swoftprocess $process) {$pname = App:: $server->getpname ();        $process->name (sprintf ('%s cronexec process ', $pname));        /** @var \swoft\task\crontab\crontab $cron */$cron = App::getbean (' Crontab ');        Swoole/httpserver $server = App:: $server->getserver ();            $server->tick (0.5 *, function () use ($cron) {$tasks = $cron->getexectasks (); if (!empty ($tasks)) {foreach ($tasks as $task) {//Diliver task Tas                    K::d eliverbyprocess ($task [' Taskclass '], $task [' Taskmethod ']);                $cron->finishtask ($task [' key ']);    }            }        }); }}

CronExecProcessAs the performer of a timed task, Swoole\Timer by 0.5s waking itself once, then 执行表 traversing once, selecting the task that needs to be performed now, by sendMessage() posting out and updating the state in the Task execution table.
The execution process is only responsible for the delivery of the task, and the actual actual execution of the task is still Task进程 TaskExecutor handled in.

The macro implementation of timed tasks is as follows:

Related recommendations:

Parsing of token generation in PHP

Parsing of URL access patterns in TP5