rabbitmq3.5.1 principle and cluster installation

Reference

http://dl528888.blog.51cto.com/2382721/1864895

http://kaibinyuan.blog.51cto.com/7304008/1610110

Reason:

Recently there is an asynchronous need to use Message Queuing, and perhaps eventually will choose Ali's ROCKETMQ

Performance

The single TPS is basically around 2000-3000

The latest RABBITMQ version is 3.6.2

Yum Installation

Configuration file

rpm-/etc/rabbitmq/

Http://www.cnblogs.com/stormli/p/rabbitmq.html

Broker: The Message Queuing server entity is simply the case.

Exchange: A message switch that specifies what rules the message is routed to and to which queue.

Queue: A message queue carrier in which each message is put into one or more queues.

Binding: Bind, which is the role of binding exchange and queue according to routing rules.

Routing key: The routing keyword, exchange messages are delivered based on this keyword.

Vhost: Virtual host, a broker can open multiple vhost, as a separate user permissions.

Producer: The message producer is the program that delivers the message.

Consumer: The message consumer is the program that receives the message.

Channel: Message channels, in each connection of the client, can establish multiple channel, each channel represents a session task

http://blog.csdn.net/u013142781/article/details/50487028

There are four concepts in AMQP that are important: virtual host, switch (Exchange), queue, and bind (binding). A virtual host holds a set of switches, queues, and bindings. Why do you need multiple virtual hosts? Very simple, RABBITMQ, the user can only be in the granularity of the virtual host permissions control. Therefore, if you need to prohibit group A from accessing the switch/queue/bindings for Group B, you must create a virtual host for A and b respectively. Each of the RABBITMQ servers has a default virtual host "/". If that's enough, then it's time to start.

Through the distributed nature of Erlang (through the Magic Cookie Authentication node) for the RabbitMQ cluster, each RabbitMQ service is a peer node, that is, each node provides services to the client connection for sending and receiving messages.

These nodes replicate the message queue structure through the RabbitMQ HA queue (mirror queue). In this scenario, 3 nodes are built, and all are disk nodes (all node states are consistent and nodes are fully equal), so long as any one node can work, the RabbitMQ cluster can provide services to the outside.

RABBITMQ System Architecture

RabbitMQ server: Also known as broker server, is a transport service that maintains a route from producer to consumer to ensure that data is transmitted in the specified manner.

Producer, the sender of the data.

Consumer, the receiver of the data.

Exchanges receives the message, forwards the message to the bound queue. The main use of 3 types: Direct, topic, fanout.

Queue rabbitmq An object that stores messages internally. A queue of the same property can be defined repeatedly, but only if the definition is valid for the first time.

Bindings binds the route between the exchanges and the queue.

Connection: is a TCP connection. Both producer and consumer are connected to RABBITMQ server through TCP.

Channel: Virtual connection. It is established in the above TCP connection. The flow of data is carried out in the channel. In other words, the general scenario is that the program starts with a TCP connection, and the second step is to establish the channel.

2. Source code Installation

Yum Remove rabbitmq-server-y

Yum-y Install Erlang

Yum-y Install gcc gcc-c++ ncurses-devel

wget http://www.erlang.org/download/otp_src_R16B01.tar.gz

Tar xvf otp_src_R16B01.tar.gz

CD OTP_SRC_R16B01

./configure && make && make install

wget http://www.rabbitmq.com/releases/rabbitmq-server/v3.5.1/rabbitmq-server-generic-unix-3.5.1.tar.gz

Tar zxvf rabbitmq-server-generic-unix-3.5.1.tar.gz

#ln-svf/usr/local/src/rabbitmq_server-3.4.3/usr/local/rabbitmq-node1

#mkdir-P/usr/local/rabbitmq-node1

Cd/opt/soft && MV Rabbitmq_server-3.5.1/usr/local

Cd/usr/local && MV rabbitmq_server-3.5.1 rabbitmq-node1

MV RABBITMQ_SERVER-3.5.1/OPT/LOCAL/RABBITMQ

Vim ~/.bash_profile

Path=/opt/mysql/bin: $PATH: $HOME/bin:/usr/lib/rabbitmq/bin/:/usr/local/rabbitmq-node1/sbin/: $PATH

SOURCE ~/.bash_profile

Enable Web Plugin

#rabbitmq-plugins Enable Rabbitmq_management #启用

Close the Web plugin

# rabbitmq-plugins Disable Rabbitmq_management

Rabbitmq-plugins Enable Rabbitmq_management

Rabbitmq-plugins List

Rabbitmqctl stop

Rabbitmq-server-detached

Rabbitmqctl Cluster_status

Scp-p 8022. Erlang.cookie [Email protected]:/root/

Ps-ef | grep RABBITMQ | Grep-v "grep"

Kill-9 54239

Rabbitmqctl join_cluster [email protected]

Rabbitmqctl Stop_app

Rabbitmqctl Start_app

Rabbitmqctl Change_password Guest Kaibin

Cat/usr/local/rabbitmq-node2/ebin/rabbit.app | grep loopback--color

{loopback_users,[<< "" >>]},

(although it can be in a more wretched way: The Ebin directory under the Rabbit.app loopback_users in the << "guest" >> Delete, or in the configuration file Rabbitmq.config to configure the item,

https://my.oschina.net/hncscwc/blog/262246

Restart RABBITMQ to access the admin console via any IP using Guest account

http://kaibinyuan.blog.51cto.com/7304008/1610110

In security considerations, guest This default user can only through localhost to log in, the other IP cannot directly use this account in order to solve this problem, you need to set the loopback_users configuration to NULL in the RABBITMQ configuration file

http://dwf07223.blog.51cto.com/8712758/1547226

Resolve RABBITMQ Remote Unreachable issues

Note: Rabbitmq from 3.3.0 is forbidden to use Guest/guest permissions through access other than localhost.

If you want to use Guest/guest for remote machine access, you need to set the loopback_users to [] in the RABBITMQ configuration file (/etc/rabbitmq/rabbitmq.config).

The full contents of the/etc/rabbitmq/rabbitmq.config file are as follows (note the following half-width period):

[{rabbit, [{loopback_users, []}]}].

RABBITMQ Web Management Interface default Guest:guest, so that is not very safe, need to change the next

[Email protected] ~]# Rabbitmqctl Change_password Guest Kaibin

Changing password for user "Guest" ...

... done.

)

Designing a mirrored queue policy

Rabbitmqctl set_policy ha-all ' ^ (?! Amq\.). * ' {' ha-mode ': ' All '} '

#rabbitmqctl set_policy ha-all "^" ' {"Ha-mode": "All"} '

#沿用3.2 environment, now we set the queue named "Hello" to synchronize to all nodes

#rabbitmqctl set_policy ha-all "Hello" ' {"Ha-mode": "All"} '

Ha-all is synchronous mode, which refers to synchronizing to all nodes, there are two other modes ha-exactly means mirroring on a specified number of nodes, the number of nodes is specified by Ha-params, and Ha-nodes is mirrored on the specified node. The node name is specified by Ha-params;

Hello is the queue name for synchronization and can be matched with regular expressions;

{"Ha-mode": "All"} means synchronization to all, the synchronization mode is different, this parameter is also different.

After executing the above command, you can view the queue page in the Web management interface, there will be a +2 tag after the node of the Hello queue, indicating that there are 2 slave nodes, and the master node is currently displayed node (xf7021 is the name of the test, Press 4-2 should be RABBITMQ (1-3)).

Mirroring mode: The message entity is actively synchronizing between nodes, rather than synchronizing when the consumer acquires it. Reduce system performance consumption bandwidth

Setting the HA mode

Rabbitmqctl Set_policy [-P <vhostpath>] [--priority <priority>] [--apply-to <apply-to>] <name> <pattern> <definition>

Name policy names

The pattern regular expression, which matches the resource, applies the set policy to the match

Definition is a policy that is set in JSON format.

Apply-to indicates what type of policy is applied to, generally queues,exchange and all, and the default is all

Priority is an integer precedence

There are three modes of Ha-mode:

All: Sync to all.

Exactly: synchronization of up to n machines. When the number of existing cluster machines is less than n, all synchronizations, greater than or equal to N, are not synchronized. n needs to be specified by an additional ha-params.

Nodes: synchronizes only to nodes that match the specified name. n needs to be specified by Ha-params. # This is set to synchronize all of the queue, you can select the specified queue on demand

Rabbitmqctl set_policy ha-all '. * ' {' ha-mode ': ' All '} '

Command detailed description

https://my.oschina.net/guol/blog/186445

Stop_app

#停止erlang the RABBITMQ application on node, but Erlang node will continue to run.

To be used in production, the following points, there are several possible

Note the place:

0. Modify the cluster Cluster_node_type

[Email protected] rabbitmq]# Rabbitmqctl Stop_app

Stopping node ' [email protected] ' ...

... done.

[[email protected] rabbitmq]# rabbitmqctl change_cluster_node_type RAM

Turning ' [email protected] ' into a RAM node ...

... done.

[Email protected] ~]# Rabbitmqctl cluster_status

Cluster Status of node ' [email protected] ' ...

[{nodes,[{disc,[' [email protected] ']},{ram,[' [email protected] '}]},

{running_nodes,[' [email protected] ', ' [email protected] '},

{partitions,[]}]

... done.

About node types (Ram |disk)

The state of the RAM node is saved in memory, and the disk node is saved in the disks that are added to the disc, as in this example [email protected] is a RAM node, [email protected],[email protected] is a RAM node, You can change the joined cluster and the node type by rabbitmqctl the cluster command after the command can be added to multiple node names, the specified node becomes a disk node

1.[[email protected] ~]# cat/etc/rabbitmq/rabbitmq-env.conf

rabbitmq_mnesia_base=/var/lib/rabbitmq///the path of the MNESIA database to be used

Path of the Rabbitmq_log_base=/var/log/rabbitmq///log

rabbitmq_plugins_dir=/usr/lib/rabbitmq/lib/rabbitmq_server-2.8.6/plugins//Plug-in Path

2. Configuring the cluster automatically

Vi/etc/rabbitmq/rabbitmq.config

Add the following content

[{rabbit,

[{cluster_nodes, {[' [email protected] ', ' [email protected] ', disc}]}].

Then start the cluster

Performed on R1 and R2

Rabbitmq-server-detached

Then view the cluster status

Rabbitmqctl Cluster_status

3. High Availability

Installing and configuring HAProxy

Install HAProxy on 192.168.1.4, and then modify the/etc/haproxy/haproxy.cfg:

Listen Rabbitmq_cluster 0.0.0.0:5672

Mode TCP

Balance Roundrobin

Server Node1 192.168.1.1:5672 check Inter rise 2 Fall 3

Server Node2 192.168.1.2:5672 check Inter rise 2 Fall 3

Server Node2 192.168.1.3:5672 check Inter rise 2 Fall 3

4. Users and Permissions

Before the formal application, we first create a vhost in RABBITMQ, add a user, and set the user's permissions. Using the RABBITMQCTL Client tool, create the "/mq_test" Vhost under the root directory:

Rabbitmqctl Add_vhost/mq_test

Create a user name "test", set the password "Test123″:

Rabbitmqctl Add_user Test test123

Set PYH user to/pyhtest this vhost has full privileges:

Rabbitmqctl set_permissions-p/mq_test Test ". *" ". *" ". *",

The following three "*" represents the Pyh user with/pyhtest configuration, write, read all permissions

conn = Amqp. Connection (host= ' localhost ', userid= ' test ', password= ' 123456 ', ssl=false,virtual_host= ' test_host ')

# Rabbitmq-plugins Enable Rabbitmq_management

# Rabbitmq-plugins List

# rabbitmqctl Add_user Admin Admin

# rabbitmqctl set_user_tags Admin Administrator

RABBITMQ Creating an Account

If it is a cluster, as long as in one host settings, the other will automatically sync.

#rabbitmqctl Add_user IOM 123456–iom for new users, 123456 for passwords

#rabbitmqctl Set_user_tags IOM administrator– to set the user as the Administrator role

#rabbitmqctl set_permissions-p/IOM ". *" ". *" ". *"

– Set up IOM User Configuration permissions, write permissions, read permissions in the/virtual host: * is the usage in regular expressions. The permissions of RABBITMQ are configured according to different virtual hosts, which may not be the same as users in different virtual hosts (Vsan).

5.rabbit_hosts=192.168.8.180:5672,192.168.8.53:5672,192.168.8.87:5672

Test the LB under the haproxy of the RABBITMQ

In the previous test code, add the

Mq_servers = [' 10.22.129.57 ', ' 10.22.129.58 ', ' 10.22.129.59 ']

Change into

Mq_servers = [' 10.22.129.53 ', ' 10.22.129.53 ', ' 10.22.129.53 ']

Execute the test code, found that three messages are sent successfully, and then even if you manually close one of the MQ, the message is still sent successfully, through Rabbitctl List_queues also can still see the message was successfully received 3.

At this point, you can see that rabbitmq-server successfully lifted the Single-point state.

6. Use service mode to start Rabbitmq-server error erlexec:home must be set

Workaround:

Use export to specify the home variable in the startup script

Export home=/opt/data/rabbitmq/

7. Most commonly used commands

View all queue information

[Email protected] ~]# Rabbitmqctl list_queues

Listing queues ...

... done.

Clear all Queues

Rabbitmqctl Reset

8. About ports

RABBIMQ 55672 15672 for monitoring plug-in ports

RABBITMQ 5672 for Data port

9. Something special to be aware of

The §cookie must be exactly the same on all nodes, so be sure to keep in mind when synchronizing.

§erlang is a host name to connect to the service, you must ensure that each host name can be ping-through. You can manually add host names and IP correspondence by editing/etc/hosts. If the hostname ping is not available, the RABBITMQ service startup will fail.

§ If the queue is a non-persistent queue, if the node that created the queue fails, the sender and receiver can create the same queue to continue working. However, if the queue is persisted, the service can only continue until the node that created the queue resumes.

§ Disk node is required to be online when the cluster metadata is changed, but all disk nodes must be online when the node joins or exits. If you do not exit disk node correctly, the cluster will assume that the node is down and do not join other nodes until the node is restored.

This article is from the "people, to have their own ideas" blog, please be sure to keep this source http://szgb2016.blog.51cto.com/340201/1882187

rabbitmq3.5.1 principle and cluster installation