RABBITMQ composition and principle introduction-3

RABBITMQ, as a mature enterprise message middleware, realizes the decoupling of interface calls between applications and improves the throughput of the system.

1.RABBITMQ Composition

• is an open source implementation of Advanced Message Queuing Protocol (AMQP) provided by LShift, written in Erlang, known for its high performance, robustness, and scalability, and therefore inherited these benefits.
• The main thing to say in AMQP is two components: Exchange and queue (there are also changes in AMQP 1.0), as shown in, the green X is Exchange, the red is the Queue, both on the Server side, also called the Broker, which is part of the Rab BITMQ is implemented, while the blue one is the client, usually there are two types of Producer and Consumer:

• RabbitMQ Server: also known as broker Server, it is not a truck that transports food, but a transport service. The exact words are rabbitmq isn ' t a food truck, it's a delivery service. His role is to maintain a route from producer to consumer, ensuring that data can be transmitted in a specified manner. But this guarantee is not a 100% guarantee, but it's enough for normal applications. Of course, for business systems, you can do a layer of data consistency guard, you can completely guarantee the consistency of the system.
• Client A & B: Also called producer, the sender of the data. Create messages and publish (send) them to a broker server (RabbitMQ). A message has two parts: payload (payload) and label (label). Payload as the name implies is the transmitted data. The label is the name of exchange or a tag that describes the payload, and RABBITMQ the label to decide which consumer to send the message to. AMQP only describes the label, and RABBITMQ determines how to use the label's rules.
• Client: Also called consumer, the receiver of the data. Consumers attach to a broker server (RabbitMQ) and subscribe to a queue. The queue is likened to a mailbox with a name. When a message arrives at a mailbox, RABBITMQ sends it to one of its subscribers, consumer. Of course, the same message may be sent to many consumer. In this message, only Payload,label has been deleted. For consumer, it is not known who sent this information. Is that the agreement itself is not supported. But of course, if producer sends a payload that contains producer information, it is another matter.

There are three concepts that need to be clarified for the correct transfer of a data from producer to consumer: exchanges, queues, and bindings.

• Exchanges is where producers publish their messages.
• Queues is where the messages end up and is received by consumers
• Bindings is how the messages get routed from the exchange to particular queues.

There are several concepts that are not indicated in the above figure, that is connection (connection), channel (channels, channel).

• Connection: is a TCP connection. Both producer and consumer are connected to RABBITMQ server through TCP. As we can see later, the starting point of the program is to establish this TCP connection.
• Channels: Virtual connection. It is established in the above TCP connection. The flow of data is carried out in the channel. In other words, the general scenario is that the program starts with a TCP connection, and the second step is to establish the channel.

• So why use the channel instead of using the TCP connection directly?

  For the OS, there is a cost to establishing and shutting down TCP connections, and the frequent establishment of closed TCP connections has a significant impact on the performance of the system, and there are limits to the number of TCP connections, which also limits the ability of the system to handle high concurrency. However, there is no such cost in establishing a channel in a TCP connection. For producer or consumer, multiple channel can be used concurrently for publish or receive. Experiments have shown that 1s of data can publish10k packets. Of course, for different hardware environment, different packet size This data certainly is not the same, but I just want to say, for ordinary consumer or producer, this is enough. If not enough, you should consider how to refine your design for split.

Exchange

Exchange is owned by Vhost. The same vhost cannot have duplicate exchange names. There are four types of exhange: Fanout,direct,header,topic.

Exchange accepts a message sent by producer and sends a message to the message Queue based on a different routing algorithm.

As you can see from the architecture diagram, Procuder Publish's message entered Exchange. Then through the "Routing keys", RABBITMQ will find out which queue the message should be placed in. The queue is also bound by this routing keys.

The binding feature of Exchange, you can bind a queue, or you can bind exchange. This looks at the specific business. If the data is bound, it needs to be distributed or reassigned, using the to exchange bindings. If you want to be processed directly, use the queue binding. If Exchange binds a queue, if the route key is not correct, it can also cause data to be unreachable and discarded. (A data can be processed by the queue, requires Exchange to bind the queue, and the Route key is equal to the bound key when the message is sent.) We send an empty data, and the Route key arbitrarily formulates the Hello, although the message is sent, but the customer gets the failure.

There are four types of exchanges:direct, Fanout,topic. Each implements a different routing algorithm (routing algorithm).

• Direct Exchange: If routing key matches, then the message is passed to the appropriate queue. In fact, when the queue is created, it will automatically bind the exchange with the name of the queue as routing key. ( forward message to routigkey specified queue )
• fanout Exchange: The queue broadcast to the response. (Forward message to all bound queues)
• Topic Exchange: pattern matching of keys, such as ab* can be passed to all ab* queue. (Forwarding messages by rules (most flexible))

Queue:

The message queue caches the message when it is not consumed normally, but when the connection between the consumer and the message queue is unobstructed, the message queue forwards the message to consumer.
The message consists of the header and body, which is a collection of various properties added by producer, including whether the message is cached, which message queue is accepted, what priority is, and so on. The body is the app data that really needs to be transferred.
The association between Exchange and the message queue is implemented through binding. Exchange generates a routing table after binding with more than one message queue, and the restriction of the message that is required to store messages in the routing table is the binding Key. When Exchange receives a message, it resolves its header to get routing key,exchange based on routing Key and exchange type to route message to message Queue.

The binding key is specified by consumer when binding exchange and Message queue, and routing key is specified by producer when sending a message, and the matching method is determined by Exchange type.
Exchange type is divided into direct (unicast), Topic (multicast), Fanout (broadcast). When direct (unicast), the Routing key must be equal to the binding key in order to match the success, when topic (multicast), the Routing key and the binding key conform to a pattern relationship is calculated matching success, when the Fanout (broadcast), Not restricted. The default Exchange type is direct (unicast).

Routing: The media that is bound between Exchange and queue becomes routing key.

message acknowledgment: The message confirms that the message acknowledgement problem is resolved and is not removed from the message system until the ACK is received.

Message Durability: messages persist, and when RABBITMQ exits or crashes, the messages in the queue are persisted. Note, however, that RABBITMQ does not guarantee that the message must not be lost, because in order to improve performance, RABBITMQ will persist the message in the memory cache until the threshold is reached to the disk, which means that if rabbitmq crashes before persisting to disk, then A small amount of data is lost, which is not unacceptable for most scenarios, and if you do need to ensure that the task is never lost, you should use a transaction mechanism.

Virtual hosts

each virtual host is essentially a RABBITMQ Server, with its own queue,exchange, and bind rule, and so on. This ensures that you can use RABBITMQ in a number of different application.

RABBITMQ's vhost is mainly used to classify different business modules. There is no information interaction between different business modules. Vhost are completely isolated from each other, and exchange and queue cannot be shared between different vhost. As a result, data between vhost cannot be shared. If you want to implement this functionality, you need to manually build the corresponding code logic between vhost.

Vhost is actually transparent between users, Vhost can be accessed by multiple users, and a user can access multiple vhost at the same time. For example, Peter can access "/" and "/vhost" at the same time.

Virtual host is a dummy concept that can hold some exchange and message Queue. A virtual host can be a server or a cluster of multiple servers. Exchange and message queues can be deployed on one or more servers, respectively.

2 Application Scenarios

Anyway RabbitMQ, or how does AMQP solve the problem, or what is its application scenario?

For a large software system, it will have a lot of components or modules or subsystems or (subsystem or Component or submodule). So how do these modules communicate? There is a big difference between this and the traditional IPC. Many of the traditional IPC are on a single system, the module coupling is very large, not suitable for expansion (Scalability), if the use of sockets so different modules can indeed be deployed to different machines, but there are still many problems to be solved. Like what:

1) How does the sender and receiver of the information maintain this connection, and how is the data lost during this period if the connection to the other party is interrupted?

2) How to reduce the coupling between sender and receiver?

3) How to get priority recipients to receive data first?

4) How do I load balance? Effectively balance the load of the receiver?

5) How to effectively send the data to the relevant recipients? In other words, the receiver subscribe different data, how to do a valid filter.

6) How can it be extensible and even send this communication module to cluster?

7) How to ensure that the recipient receives the complete, correct data?

The AMDQ protocol solves the above problems, and RABBITMQ implements AMQP.

RABBITMQ composition and principle introduction-3