about how time-consuming operations are handled guess/I use the queue operation

Quote: In many Web applications, we handle our tasks in a synchronous way, but there are often business demands that an operation might be time-consuming, or that there is a lot of uncertainty (such as a payment operation that waits for third-party results to be notified). In this business scenario, it may not be reasonable to use a synchronous approach to the operation. What we think about is that it is necessary to use synchronization to return temporary results, and then to process them by asynchronously notifying the final result.

So how do you deal with this kind of scenario?

For languages that have their own ability to handle asynchronous operations, such as Java, it is only necessary to throw a thread into the code and specify a callback handler.

For languages that do not handle asynchronous operations on their own (such as PHP), it is relatively troublesome, generally need to rely on some of the running process in the background, it will always go to detect whether there is a need to deal with things, if any, according to demand, followed by the operation. Like this operation, we can call that message processing. Then, when the time-consuming operation is encountered, we need to immediately store the message in the message queue so that the background process can read the message!

There are a lot of ready-made Message Queuing software such as: Rabbitmq,kafka, if there is a need, you can choose one to use, this article only to explain some basic principles and personal practice.

Because I use the PHP language, it is only the PHP feature to express their own way of processing. For languages that have their own processing power, they can handle a variety of concurrency problems on their own, but for languages that require Cron scripting assistance, more action is required!

1. How do I put messages in a queue?

Since the operation is time-consuming, we can think about the time-consuming operation, first processing, and then handing over the subsequent work to the server, then how do we store these things to operate? Methods should be many, such as storing to a file, storing it in a database, and storing it in a cache. Of course, the best way to do this is to store it in cache, and with a caching tool like Redis or memcache, you can easily implement push and pop operations. But the message may just be some string, how to express clearly what needs to be done? This, of course, is also a lot of, you define the structure of good news, such as: {MSGTYPE:AA, params: [{p1:2, p2:3}], addtime:2016-05-15 10:30:03, from:[{ip:1.1.1.1, Who:lee }]}, then serialize or JSON the message and save it!

2. How do I handle messages?

How do I deal with messages when they are queued? There are generally two kinds: 1. Write a dead loop, always go to detect the message content, after detection, according to the message type, immediately do the corresponding processing. 2. Use cron script to check the message regularly, have the processing, do not quit! In fact, there is a third, using Cron to execute the script on a regular basis, but a single execution will need to process all the messages, and then return, improve efficiency! (The message must be deleted or stored elsewhere after each message is processed)

3. How do I avoid concurrent execution?

The message should be executed only once, and if executed more than once, the expected result is exceeded, so be aware that multiple processes execute a message at the same time, mainly in two ways, 1. The program creates the file lock itself, detects the presence of the lock before execution, does not create it, deletes it after execution, but it is important to note that the program terminates unexpectedly (if terminated unexpectedly, the lock will persist and subsequent execution will never be performed). 2. Let cron generate a lock and control the file to be executed by only one process at a time;

4. Precautions?

If the message is stored in the cache only, after execution is done, it is deleted, it is fatal for both troubleshooting and recovery, so it is generally necessary to store it in a backup location before deleting it, or to save the message as a log!

In a nutshell, Message Queuing is a process of message push, a message execution. Other, just to ensure the implementation of the process of various accidents, various recovery, all kinds of re-operation! (I can't seem to find anything else to say about me, hahaha ...) ）

about how time-consuming operations are handled guess/I use the queue operation