PHP Message Queuing

This article is mainly to share with you the PHP message queue in detail, I hope to help everyone, first we first understand what is Message Queuing.

1. What is Message Queuing

Message Queuing (English: Message queue) is an inter-process communication or a way of communicating between different threads of the same process

2. Why use Message Queuing

Message Queuing technology is a technique for exchanging information among distributed applications. Message Queuing can reside in memory or on disk, and queues store messages until they are read by the application. With Message Queuing, applications can execute independently, they do not need to know each other's location, or do not need to wait for the receiving program to receive this message before continuing execution.

3. What is the occasion for using Message Queuing

You first need to figure out the difference between Message Queuing and remote procedure calls, and when many readers consult me, I find that they need RPC (remote procedure Call) instead of Message Queuing.

Message Queuing has synchronous or asynchronous implementations, usually we use Message Queuing asynchronously, and remote procedure calls are mostly synchronous.

What is the difference between MQ and RPC? MQ typically passes an irregular protocol, which is defined by the user and implements store forwarding, whereas RPC is usually a private protocol, and the call procedure returns the result.

4. When to use Message Queuing

Synchronous requirements, remote procedure Call (PRC) is better for you.

asynchronous requirements, Message Queuing is better for you.

Currently, many Message Queuing software supports RPC functionality, and many RPC systems can also be called asynchronously.

Message Queuing is used to implement the following requirements

Store and forward

Distributed transactions

Publish a subscription

Content-based routing

Point-to-point connections

5. Who is responsible for handling Message Queuing

As a general practice, if a small project team can have a person implement, including message push, receive processing. If a large team typically defines a good message protocol and then each develops its own part, such as a team responsible for writing the Push Protocol section, the other team is responsible for writing the receive and processing sections.

So why don't we talk about the framing of message queues?

There are several benefits of framing:

Developers do not have to learn Message Queuing interfaces

Developers do not need to care about message push and receive

Developers push messages through a unified API

The developer's focus is on implementing business logic functions

6. How to implement the message queue framework

The following is an SOA framework developed by the author, which provides three interfaces, SOAP,RESTFUL,AMQP (RabbitMQ), that understand the framework and that you can easily expand further, such as adding XML-RPC, ZEROMQ, and so on.

Https://github.com/netkiller/SOA

This article speaks only about the Message Queue Framework section.

6.1. Daemon process

The message queue Framework is a local application (a command-line program) and we need to implement the daemon in order for him to run in the background.

https://github.com/netkiller/SOA/blob/master/bin/rabbitmq.php

Each instance processes a set of queues, the instantiation needs to provide three parameters, $queueName = ' queue name ', $exchangeName = ' interchange name ', $routeKey = ' route '

$daemon = new \framework\rabbitdaemon ($queueName = ' email ', $exchangeName = ' email ', $routeKey = ' email ');

The daemon needs to run with the root user, switch to a normal user after running, and create a process ID file to use when the process is stopped.

Daemon Core Code https://github.com/netkiller/SOA/blob/master/system/rabbitdaemon.class.php

6.2. Message Queuing protocol

A message protocol is an array that serializes or converts an array to a Message Queuing server, using a JSON-formatted protocol.

$msg = Array (' Namespace ' = ' Namespace ', "Class" and "e-mail", "Method" = "smtp", "Param" = = Array ($mail, $ Subject, $message, null));

Post-Serialization Protocol

{"Namespace": "single", "Class": "Email", "Method": "SMTP", "Param": ["netkiller@msn.com", "Hello", "Testhelloworld", NULL]}

The use of JSON format is considered universal, so that the push end can be used in any language. If compatibility is not considered, it is recommended to use binary serialization, such as Msgpack, which is more efficient.

6.3. Message Queuing processing

Message Queuing processing Core code

https://github.com/netkiller/SOA/blob/master/system/rabbitmq.class.php

So the processing of the message is done in the following section of code

$this->queue->consume (function ($envelope, $queue) {$speed = Microtime (true); $msg = $envelope->getbody (); $ result = $this->loader ($msg); $queue->ack ($envelope->getdeliverytag ()); Send ACK response manually//$this->logging->info (". $msg. $result) $this->logging->debug (' Protocol: '. $msg. '); $ This->logging->debug (' Result: '. $result. ' $this->logging->debug (' Time: '). (Microtime (True)-$speed).);

Public Function loader ($msg = NULL) is responsible for disassembling the protocol, then loading the corresponding class file, passing parameters, running methods, and feedback results.

Time can output how long it takes the program to run, which is useful for post-optimization.

Tips

Loader () can be further optimized by using multithreading to submit tasks to the thread pool each time it is called loader, so that Message Queuing can be processed in multithreading.

6.4. Testing

Test Code https://github.com/netkiller/SOA/blob/master/test/queue/email.php

<?php$queuename = ' example '; $exchangeName = ' email '; $routeKey = ' email '; $ mail = $argv [1]; $subject = $argv [2]; $message = Empty ($argv [3])? ' Hello world! ': '. $argv [3]; $connection = new Amqpconnection (Array (' host ' = ' 192.168.4.1 ', ' port ' = ' 5672 ', ' vhost ' = '/', ' login ' = ') Guest ', ' password ' = ' guest '), $connection->connect () or Die ("Cannot connect to the broker!\n"); $channel = new Amqpchannel ($connection), $exchange = new Amqpexchange ($channel); $exchange->setname ($exchangeName); $queue = new Amqpqueue ($channel); $queue->setname ($queueName); $queue->setflags (amqp_durable); $queue Declarequeue (); $msg = Array (' Namespace ' = ' Namespace ', "Class" = "Email", "Method" = "smtp", "Param" = Array ($mail, $subject, $message, null)) $exchange->publish (Json_encode ($msg), $routeKey);p rintf ("[x] Sent%s \ r \ n" , Json_encode ($msg)); $connection->disconnect (); 

Only a small number of test and demonstration programs are given here, if you have any questions, please go to the Profanity Group, or ask the public.

7. Multithreading

The above Message Queue Core code is as follows

$this->queue->consume (function ($envelope, $queue) {$msg = $envelope->getbody (); $result = $this->loader ($ msg); $queue->ack ($envelope->getdeliverytag ());});

This code production environment used for half a year, found that the efficiency is low. Some business fields are very fast, but they take longer to process and tend to queue up.

Increasing multithreading can make more efficient use of hardware resources and improve business processing power. The code is as follows

<?phpnamespace framework;require_once (__dir__. ') /autoload.class.php '), class Rabbitthread extends \threaded {private $queue;p ublic $classspath;p rotected $msg;p ublic function __construct ($queue, $logging, $msg) {$this->classspath = __dir__. ' /.. /queue '; $this->msg = $msg; $this->logging = $logging; $this->queue = $queue;} Public Function Run () {$speed = Microtime (true); $result = $this->loader ($this->msg); $this->logging->debug (' Result: '. $result. ' $this->logging->debug (' Time: '). (Microtime (True)-$speed).} privatepublic function Loader ($msg = null) {$protocol = Json_decode ($msg, true); $namespace = $protocol [' namespace '];$ class = $protocol [' class ']; $method = $protocol [' method ']; $param = $protocol [' param ']; $result = null; $classspath = $this- >classspath. ' /'. $this->queue. ' /'. $namespace. ' /'. Strtolower ($class). '. class.php ', if (Is_file ($classspath)) {require_once ($classspath);//$class = Ucfirst (substr ($request _uri, Strrpos ($ Request_uri, '/') +1)); (Class_exists ($class)) {if (method_exists ($class, $method)) {$obj = new $class, if (! $param) {$tmp = $obj. $method (); $result = Json_encode ($tmp ); $this->logging->info ($class. '. $method. ' ()');} else{$tmp = Call_user_func_array (Array ($obj, $method), $param); $result = (Json_encode ($tmp)); $this->logging-> Info ($class. '. $method. ' ("'. Implode ('", "', $param) ') ');}} else{$this->logging->error (' Object ' $class. ' and '. $method. ' is not exist. ');}} else{$msg = sprintf ("Object is not exist. (%s) ", $class); $this->logging->error ($msg);}} else{$msg = sprintf ("Cannot loading interface! (%s) ", $classspath); $this->logging->error ($msg);} return $result;}} Class RabbitMQ {Const LOOP = 10;protected $queue;p rotected $pool;p ublic function __construct ($queueName = ', $exchangeNam E = ', $routeKey = ') {$this->config = new \framework\config (' Rabbitmq.ini '); $this->logfile = __dir__. ' /.. /log/rabbitmq.%s.log '; $this->logqueue = __dir__. ' /.. /log/queue.%s.log '; $this->logging = new \framework\log\logging ($this->logfile, $debug =true); //. H:i:s$this->queuename= $queueName, $this->exchangename= $exchangeName, $this->routekey= $routeKey; $this->pool = new \pool ($this->config->get (' pool ') [' thread ']);} Public Function Main () {$connection = new \amqpconnection ($this->config->get (' rabbitmq ')); try {$connection- Connect (), if (! $connection->isconnected ()) {$this->logging->exception ("Cannot connect to the broker!". PHP_EOL);} $this->channel = new \amqpchannel ($connection), $this->exchange = new \amqpexchange ($this->channel); $this- >exchange->setname ($this->exchangename); $this->exchange->settype (Amqp_ex_type_direct); Direct type $this->exchange->setflags (amqp_durable); Long-lasting? $this->exchange->declareexchange () $this->queue = new \amqpqueue ($this->channel); $this Queue->setname ($this->queuename); $this->queue->setflags (amqp_durable); Long-lasting? $this->queue->declarequeue (); $tHis->queue->bind ($this->exchangename, $this->routekey); $this->queue->consume (function ($ Envelope, $queue) {$msg = $envelope->getbody (); $this->logging->debug (' Protocol: '. $msg. ');//$result = $ This->loader ($msg); $this->pool->submit (New Rabbitthread ($this->queuename, New \framework\log\logging ( $this->logqueue, $debug =true), $msg) $queue->ack ($envelope->getdeliverytag ()); }); $this->channel->qos (0,1);} catch (\amqpconnectionexception $e) {$this->logging->exception ($e->__tostring ());} catch (\exception $e) {$this->logging->exception ($e->__tostring ()); $connection->disconnect (); $this- >pool->shutdown ();}} Private Function Fault ($tag, $msg) {$this->logging->exception ($msg); throw new \exception ($tag. ': '. $msg);} Public Function __destruct () {}}