Python Road Day11 "RabbitMQ, Redis, Mysql"

Outline

1.RabbitMQ

2.Redis

3.Mysql

1.RabbitMQ Message Queuing

1.1 RABBITMQ Introduction

AMQP, Advanced message Queuing Protocol, is an open standard for application-layer protocols designed for message-oriented middleware. Message middleware is mainly used for decoupling between components, the sender of the message does not need to know the existence of the message consumer, and vice versa.
The main features of AMQP are message-oriented, queue, routing (including point-to-point and publish/subscribe), reliability, and security.
RABBITMQ is an open-source AMQP implementation that is written in Erlang and supported by a variety of clients such as Python, Ruby,. NET, Java, JMS, C, PHP, ActionScript, XMPP, stomp, etc., and support Ajax. It is used to store and forward messages in distributed system, which is very good in ease of use, extensibility, high availability and so on.
The following will focus on some of the basic concepts in RABBITMQ, and understand these concepts as the basis for using good RABBITMQ.

1.2 Installing the RABBITMQ and Python pika modules

1.2.1 Installation RABBITMQ

(1) Installing the Erlang platform (RABBITMQ dependent platform)

1. Installing dependent files
Yum Install Ncurses-devel

2. Download the source file
wget http://www.erlang.org/download/otp_src_19.1.tar.gz
If it fails, go to address: http://erlang.org/download/to download manually

3. Extract the source file compression package
TAR-XVF otp_src_17.1.tar.gz
(Tar parameter meaning: bz2 format with j;gz format with Z;c is created; X is decompressed; v is details; F is the specified file)

4. Enter the extracted directory
CD otp_src_19.1

5. Execute the following command in turn:
The./configure-prefix=/opt/erlang will start compiling the installation will be compiled to/opt/erlang
Make && make install

6. Modify the/etc/profile file to add the following environment variables:
cd/etc/
#set Erlang Environment
Export path= $PATH:/opt/erlang/bin
Source profile makes the file effective (use export to see if there are any environment variables just added in the path)

7. After the installation is completed, the ERL will see if the Eshell can be opened with ' Halt (). ' Exit, note: "." It's the Terminator of Erlang.

(2) Installation RABBITMQ

Wget-c http://www.rabbitmq.com/releases/rabbitmq-server/v3.6.0/rabbitmq-server-3.6.0-1.noarch.rpm

RPM-IVH--nodeps rabbitmq-server-3.6.0-1.noarch.rpm

1.2.2 Installation Pika

Pip install Pika or Easy_install Pika

1.3 Simplest send/Receive Message Queuing model

Producer

1 #!/usr/bin/env Python32 #-*-coding:utf-8-*-3 4 ImportPika5 6Connection =Pika. Blockingconnection (Pika. Connectionparameters (7     'localhost') # is equivalent to establishing a socket connection8Channel =Connection.channel ()9 #declaring a queueTenChannel.queue_declare (queue='Hello') One #RabbitMQ A message can never is sent directly to the queue, it always needs to go through an exchange. AChannel.basic_publish (exchange="', -routing_key='Hello', -body='Hi there! '. Encode ("Utf-8")) the Print("send ' Hello! '") -Connection.close ()

Consumer

1 #!/usr/bin/env Python32 #-*-coding:utf-8-*-3 ImportPika4 5Connection =Pika. Blockingconnection (Pika. Connectionparameters (6     'localhost'))7Channel =Connection.channel ()8 9 #May ask why we declare the queue again? We had already declared it in our previous code.Ten #we could avoid so if We were sure that the queue already exists. For example if send.py program One #was run before. But we ' re not yet sure which program to run first. In such cases it ' s a good A #practice to repeat declaring the queue in both programs. -Channel.queue_declare (queue='Hello') -  the  - defCallback (ch, method, properties, body): -     Print("Roger:%r"% Body.decode ("Utf-8")) -  + Channel.basic_consume (Callback, -Queue='Hello', +no_ack=True) A Print('wait for ... ') atChannel.start_consuming ()

Note The English comments in the code, especially why you declared the queue again ...

1.4 Polling principle

1.3 If you run two consumer, respectively, remember Consumer1, Consumer2, then producer first message is Consumer1 received, the second hair is Consumer2 received, the third hair is Consumer1 received ...... That is, RABBITMQ sends the message to the consumer end in turn.

1.5 Persistence of messages

Producer

1 #!/usr/bin/env Python32 #-*-coding:utf-8-*-3 ImportPika4 5Connection = Pika. Blockingconnection (Pika. Connectionparameters ("localhost"))#equivalent to establishing a socket connection6Channel = Connection.channel ()#Define a pipeline7 #declaring a queue8Channel.queue_declare (queue="Hello2", durable=true)#Durable=true is to persist this queue, if RABBITMQ hangs, the queue is still there;9                                                       #Messages in the queue are not persisted, the message is lostTenChannel.basic_publish (exchange="", Onerouting_key="Hello2", Abody="hi,how is you?", -properties=Pika. Basicproperties ( -delivery_mode=2,))#Properties=pika. Basicproperties (delivery_mode=2,) This is the message persistence in the queue the Print("sent a sentence ... ") -Connection.close ()

In the code above, line 8th is just a queue persistence, and if RABBITMQ hangs, the queue is still there, but if the messages in the queue are not persisted, the message is lost.

1.6 Message Fair distribution

If rabbit in order to send the message to the individual consumers, regardless of consumer load, it is likely to occur, a machine is not high-profile consumers piled up a lot of message processing, while the high-profile consumers have been very easy. To solve this problem, you can configure perfetch=1 on each consumer side, meaning to tell rabbitmq not to send me any new messages until I have finished with the current consumer message.

1.7 Message Publish/subscribe (publish/subscribe)

1.8 Selectively receiving messages (exchange type = direct)

Python Road Day11 "RabbitMQ, Redis, Mysql"