Distributed Message Queue Based on Redis

Distributed Message Queue Based on Redis

1. Why do I need a message queue?
When factors such as the speed or stability of "production" and "consumption" are inconsistent in the system, a message queue is required as an abstraction layer to bridge the differences between the two parties.

For example, the business system triggers a text message sending application, but the text message sending module cannot keep up with the speed. You need to save messages that are too late to be processed to buffer the pressure.
For another example, it is costly to place an order in the remote system. due to factors such as the network, the order is unstable and can be sent together in batches.
Another example is that the Interaction module is connected to the e-commerce system from to and is disconnected from the internal ERP system. Connect to ERP from to, and disconnect with e-commerce systems.
For another example, the waiter can order food quickly and the cook can cook slowly.
For another example, there are many people working in a bank, but there are few service providers.
Wait in line.

2. What are the benefits of using MQ?
2.1. Improve system response speed
When a message queue is used, the producer party can immediately return and respond to the user by throwing the message into the queue. You do not need to wait for the processing result.

The processing result can be obtained by the user later, for example, the hospital orders. You can also subscribe to the producer (for example, by leaving a mobile phone number or enabling the producer to implement the listener interface and add the producer to the listener Queue). A notification is returned. Get conventions and place the results somewhere without notice.

2.2. Improve System Stability
Consider the situation where the E-commerce system places an order and sends data to the production system.
The network between the E-commerce system and the production system may be disconnected, and the production system may suspend services for maintenance or other reasons.

If no message queue is used, E-commerce system data is released, and customers cannot place orders, affecting business development.
The two systems should not be so tightly coupled. The message queue should be used for decoupling. It also makes the system more robust and stable.

3. Why is distributed?
3.1 multi-system collaboration requires distributed
The data in the message queue must be shared among multiple systems to realize its value.
Therefore, distributed communication and collaboration mechanisms must be provided.

3.2 distributed deployment environment in a single system
Within a single system, most of them are cluster environments for better performance and to avoid spof.
In the cluster environment, applications run in multiple JVMs of multiple servers, and data is stored on multiple nodes of various types of databases or non-databases.
Distributed solutions are required to meet the needs of multi-node collaboration.

4. What problems should be solved in a distributed environment?
4.1 concurrency Problems
Good concurrency control is required. Ensure "thread security".

Do not ship an order twice. Do not issue A ticket from Customer A. The shipping is sent to Customer B.

4.2 simple and unified operation mechanism
A simple, semantic, and business-independent access method should be defined, and a proper and secure unified access method should be defined.

4.3 Fault Tolerance
Control spof to ensure data security.

4.4. Horizontally scalable
Convenient resizing.

5. How to implement it?
There are too many mature message queue middleware Products, and the family cannot be used for backup.

Proven products, interface specifications, and high scalability.

Considering the factors of the business environment, the legacy of the organizational process, the implementation and O & M considerations, the technical route considerations, and the development personnel situation, Redis is the most feasible choice.

What should we do?
Listen to the next decomposition.

Install and test Redis in Ubuntu 14.04

Redis cluster details

Install Redis in Ubuntu 12.10 (graphic explanation) + Jedis to connect to Redis

Redis series-installation, deployment, and maintenance

Install Redis in CentOS 6.3

Learning notes on Redis installation and deployment

Redis. conf

For more details, please continue to read the highlights on the next page:

1 2 3 4 Next Page