"Go" You may not need Message Queuing

I'm a minimalist, and I don't like to complicate software prematurely or unnecessarily. Adding components to a software system is a way to increase complexity significantly. Let's get the message queue for an example.

Message Queuing is a system that gives you the ability to get fault-tolerant, distributed, decoupled architectures. On paper, it looks good.

Perhaps there are many scenarios in your application for Message Queuing. You can look at this article on the merits of Message queuing and see what the right scenarios are. But do not use it easily because it is "too good to be decoupled". Let's look at an example-you want your mail to be sent and the order processed to be decoupled from each other.

So you send a message to the message queue, and then the mail processing system takes out the message and sends the message. How do you do it in a standalone single-classpath application? Make your order Processing service dependent on a mail service and then call the SendEmail () method instead of the Sendtomq () method. If you use Message Queuing, you need to define a message format that both systems can recognize, and if you don't use Message Queuing, you have to define a method signature. What are the essential differences between them? Not really.

But you may have other consumers who want to take extra care of a given message? This is really possible, not just for the project we're talking about here. Although it is possible, it may not be worth compared to adding another method call. Coupling? Yes. But there is nothing inconvenient about this coupling.

What should I do with peak traffic? You can put requests into a persistent queue through Message Queuing, and then process them together. This is a very useful feature, but it is also limited by a few factors-is your request being processed in the UI, or does it require an immediate response? The thread pool of the Serlvet container can be considered to be a queue in some way, and the user will eventually get a response, but it will have to wait (if the thread's timeout is too short, the request may be lost).

You can use a memory queue to store those heavier requests (which have to be processed in the background of the UI). Note, however, that your queue is not the default high-availability. For example, if a message queue node is hung up, your message is lost. Therefore, instead of using the memory queue within the application node, you use a message queue, which may not have any advantage.

Message Queuing allows us to do asynchronous processing-which is really a useful feature. You don't want to do something very heavy when the user is waiting. But you can also use a memory queue, or simply start a new thread (such as spring's @async annotations). So there's another question--is there a problem if the message is lost? If the node you are applying processing requests to hangs up, can you restore it? You will find that this happens often, and it is difficult to guarantee the correctness of the function if you do not guarantee that all messages will be processed. Therefore, it is preferable to only perform asynchronous processing of the heavier calls.

Put the message in the queue for another component to handle, and for this scenario, if the message is lost is unacceptable, there is a very simple solution-the database. You can store a piece of processed=false data in a database. Then run a dispatch job that picks up all the unhandled records and processes them asynchronously. When processing is complete, set the token to true. I often use this method, including in some large online systems, and it works very well.

This allows you to continually extend your application nodes as long as they do not have any persisted state in memory. Whether or not you use Message Queuing (a temporary memory processing queue is not a persisted state).

Why do I have to provide some alternatives for a frequently used message queue? Because if you choose it for inappropriate reasons, Message Queuing can be a burden. They are not as easy to use as imagined. First, it has a learning curve. In general, the more components you integrate, the more likely you are to have problems. Second, there is a cost of setup and configuration. For example, when Message Queuing needs to run in a cluster, such as multiple data centers, this becomes complicated.

High availability is not something that comes up--it doesn't open by default. And how is your application node connected to Message Queuing? Through a refreshed connection pool, or with a short life cycle DNS record, or through a load balancer? Your queue may also have many configuration items, the size of how much, how the behavior (consumers need not to acknowledge acceptance, to notify the failure of processing, multiple consumers can get the same message, the message there is no TTL, and so on) there is also the network and message delivery overhead, Especially now everyone likes to use XML or JSON to transmit messages. If you use the message queue excessively, it will increase your system's latency.

Last but not least-using Message Queuing can make problem tracking extremely difficult in the event of a problem. You can't see the call hierarchy in the IDE, because once you send a message to the queue, you have to find out where it's handled. It's not as simple as it sounds. You see, it adds a lot of complexity to you, and a lot of things to watch out for.

In general, Message Queuing is very useful in some contexts. When they do, I also use them in the project--for example, we don't want to lose the message, but we want to be able to handle it quickly. I've seen it in some less common scenarios, such as having only one application node to consume, regardless of which node the message was posted to. You can also look at the problem on the StackOverflow. There are also scenarios where you do need to communicate in multiple languages, or if your data flow is too complex to use new message consumers instead of adding new method calls.

What I want to say is the old-fashioned truth of "killing chickens with sledgehammer". If you are not sure that there is no other method that is easier to manage and maintain, be sure to use Message Queuing, it is best not to use it. Don't use it because "in case it works"-only if you really feel you need it. Because it's possible, like this one here, that Message Queuing is not really necessary.

English Original: You Probably Don ' t need a Message Queue

"Go" You may not need Message Queuing