Python operations Redis, RabbitMQ, SQLAlchemy, Paramiko, MySQL

First, Redis

Redis is an open source API that is written in ANSI C, supports the web, can be persisted in memory, key-value databases, and provides multiple languages.
Redis is a key-value storage system. Similar to memcached, it supports storing more value types, including string (string), list (linked list), set (set), Zset (sorted set-ordered collection), and hash (hash type). These data types support Push/pop, Add/remove, and intersection-set and difference sets, and richer operations, and these operations are atomic. Based on this, Redis supports sorting in a variety of different ways. As with memcached, data is cached in memory to ensure efficiency. The difference is that Redis periodically writes the updated data to disk or writes the modified operation to the appended record file, and Master-slave (Master-Slave) synchronization is implemented on this basis.
Redis is a high-performance Key-value database. The emergence of Redis, to a large extent, compensates for the lack of memcached such key/value storage, in some cases can be a good complement to the relational database. It provides clients such as Java,c/c++,c#,php,javascript,perl,object-c,python,ruby,erlang, which is convenient to use.
Redis supports master-slave synchronization. Data can be synchronized from the primary server to any number of slave servers, from the server to the primary server that is associated with other slave servers. This enables Redis to perform single-layer tree replication. You can write to the data intentionally or unintentionally. Because of the full implementation of the publish/subscribe mechanism, you can subscribe to a channel and receive a complete message release record from the master server when the tree is synchronized anywhere from the database. Synchronization is helpful for the scalability and data redundancy of read operations.
1.Redis Installation

$ wget http://download.redis.io/releases/redis-3.0.3.tar.gz
$ tar xzf redis-3.0.3.tar.gz
$ CD redis-3.0.3
$ make
$ src/redis-server
# Open service, default port number: 6379

2.Package Installation

$ sudo pip install Redis
Or
$ sudo easy_install redis
or from source
$ sudo python setup.py install
3. Use

>>> Import Redis
>>> r = Redis. Strictredis (host= ' localhost ', port=6379, db=0)
>>> r.set (' foo ', ' Bar ')
True
>>> r.get (' foo ')
' Bar '
A pipeline (pipeline) is a subclass of a base class in which Redis buffers multiple server commands in a single request. It greatly improves the ability to perform bulk commands by reducing the repeated TCP database packets between servers-clients.

>>> p.set (' Hello ', ' Redis '). P.sadd (' Faz ', ' Baz '). INCR (' num '). Execute ()
When there are many types of document objects, the contents of the document are different, (that is, "table" does not have a fixed column), you can use hash to express.

Copy Code
>>> r.hset (' Users:jdoe ', ' name ', ' John Doe ')
1L
>>> r.hset (' users:jdoe ', ' email ', ' [email protected] ')
1L
>>> r.hset (' Users:jdoe ', ' phone ', ' 1555313940 ')
1L
>>> R.hincrby (' Users:jdoe ', ' visits ', 1)
1L
>>> r.hgetall (' Users:jdoe ')
{' phone ': ' 1555313940 ', ' name ': ' John Doe ', ' visits ': ' 1 ', ' email ': ' [email protected] '
>>> R.hkeys (' Users:jdoe ')
[' name ', ' email ', ' phone ', ' visits ']

Second, RabbitMQ

RABBITMQ equivalent to a message agent, he finished receiving and forwarding the message function, you can think of it as a post office, play a role in the relay. RABBITMQ will use a number of technical terms, as follows:

>producer: The program used to send messages is called a Producer, and we use ' P ' to denote:

>queue: Queue, equivalent to the role of the mailbox, he has been created after the survival and RABBITMQ, although the message can flow between your programs, but in this middle of the process, the message must exist in the queue, queue no size limit, you can save countless messages, (If you have to set aside 1GB of hard disk space), he is the equivalent of an unrestricted cache. Multiple producer can send messages to the same queue, and multiple consumer can also receive messages from the same queue. A queue can be represented by the following diagram, which is the name of the queue.

>consumer: A program to receive messages called Consumer, we use the following diagram to represent:

Note that for producer and consumer you may not be on the same host, and subsequent posts will be introduced.

2.2 Two-point transmission "Hello world!" ”

Requires two programs, one for sending "Hello world! ", a program used to receive" Hello world! and print to the top of the screen. The model is as follows:

We create a queue named Hello. Producer sends a message to the Queue,consumer of hello to receive the message from the queue named Hello.

2.3sending (send code implementation)

The model is as follows:

First we need to write a send.py program, which will send a message to the queue, the first thing we do is to establish a connection with Rabbitmq-server, the code is as follows:

Import Pika
Connection = Pika. Blockingconnection (Pika. Connectionparameters (host= ' localhost '))
Channel = Connection.channel ()
If we want to establish a connection with a different host, simply change the ' localhost ' to an IP address.

Next, we need to determine the presence of the queue that accepts the message, and if we send the message to a nonexistent queue, then RABBITMQ automatically discards the message, let's create a Message Queuing queue named ' Hello '

Channel.queue_declare (queue= ' Hello ')
In RABBITMQ, a message cannot be sent directly to the queue, it needs to go through an exchange, and a subsequent post will talk about, now we just set Exchange to an empty string.

Exchange is special, and he will identify our message to which message queue Queue,queue's name needs to be identified by Routing_key, and exchange determines which Message Queuing queue the message is sent to by Routing_key. The code is as follows:

Channel.basic_publish (exchange= ", routing_key= ' hello ', body= ' Hello world! ')
Print "[X] Sent ' Hello world! '"
Before exiting the program, we need to clean up the cache and determine our message "Hello world! "Really sent to RABBITMQ, we can close the connection to complete, the code is as follows:

Connection.close ()
The complete send.py code is as follows:

Import Pika
Connection = Pika. Blockingconnection (Pika. Connectionparameters (
host= ' localhost '))
Channel = Connection.channel ()
Channel.queue_declare (queue= ' Hello ')
Channel.basic_publish (exchange= ",
routing_key= ' Hello ',
body= ' Hello world! ')
print "[x] Sent ' Hello world! '"
Connection.close ()
2.4Receiving (Receive code implementation)

The model is as follows:

Our receiving program receive.py will receive the message and print it on the screen.

Also first, we need to establish a connection with Rabbitmq-server, the code and send.py are basically the same, the code is as follows:

Import Pika
Connection = Pika. Blockingconnection (Pika. Connectionparameters (host= ' localhost '))
Channel=connection.channel ()
Next, as before, to determine the existence of a queue of queues, to create a queue with Queue_declare (), we can run many times for this command, but only a queue with the name Hello is present. The code is as follows:

Channel.queue_declare (queue= ' Hello ')
Perhaps you will ask why we want to create a queue with the name Hello again, before we have established Ah, of course, if we make sure the queue is already there, we can not add this code. For example, if the send.py program has been run before, but we are not sure which program to run first, it is necessary to declare the queue in two programs at the same time.

For receive.py, we are going to define a callback function that, when we receive the message, the Pika library calls the callback function, and in our program, the callback function prints the received message on the screen. The function code is as follows:

DEF callback (Ch,method,properties,body):
print "[x] Received%r"% (body)
Next we need to tell RABBITMQ that the function of this special function, callback (), is to receive messages from the queue of Hello, which is the following code:

Channel.basic_consume (callback,queue= ' Hello ', no_ack=true)
For No_ack in the code, the following blog will be mentioned.

Finally, we open a thread that never stops waiting for the message and runs the callback () function when needed. The code is as follows:

print ' [*] waiting for message. To exit Press CTRL + C '
Channel.start_consuming ()
The complete receive.py code is as follows:

Import Pika
Connection = Pika. Blockingconnection (Pika. Connectionparameters (
host= ' localhost '))
Channel = Connection.channel ()
Channel.queue_declare (queue= ' Hello ')
print ' [*] waiting for messages. To exit Press CTRL + C '
DEF callback (ch, method, properties, body):
print "[x] Received%r"% (body,)
Channel.basic_consume (Callback,
Queue= ' Hello ',
No_ack=true)
Channel.start_consuming ()
2.5 Code Testing

First open a command-line window, run the send.py code, run and result as follows:

$: Python send.py
[x] Sent ' Hello world! '
send.py stops every time it runs, let's run the received code receive.py, and the results are as follows:

$: Python receive.py
[*] Waiting for messages. To exit Press CTRL + C
[x] Received ' Hello world! '
You will see that received.py runs up and does not stop, always waiting for the incoming message, if you want to stop, CTRL + C.

Continuing to run send.py in a new command-line window, the command-line window running receive.py will continue to output the appropriate information.

2.6 Section rabbitmq-server command-line Action commands

1) View the names of each queue and the number of messages in the queue

$: sudo rabbitmqctl list_queues
For example

$: sudo rabbitmqctl list_queues
Listing queues ...
Hello 0
... done.
2) View the names of each exchange

$: sudo rabbitmqctl list_exchanges

Third, Paramiko

1, Sshclient

Used to connect to a remote server and execute basic commands

Connect based on user name password:

Import Paramiko

# Create an SSH object
SSH = Paramiko. Sshclient ()
# Allow connections to hosts that are not in the Know_hosts file
Ssh.set_missing_host_key_policy (Paramiko. Autoaddpolicy ())
# Connection Server
Ssh.connect (hostname= ' c1.salt.com ', port=22, username= ' Wupeiqi ', password= ' 123 ')

# Execute Command
stdin, stdout, stderr = Ssh.exec_command (' df ')
# Get command results
result = Stdout.read ()

# Close Connection
Ssh.close ()

Copy Code
Import Paramiko

Transport = Paramiko. Transport (' hostname ', 22)
Transport.connect (username= ' Wupeiqi ', password= ' 123 ')

SSH = Paramiko. Sshclient ()
Ssh._transport = Transport

stdin, stdout, stderr = Ssh.exec_command (' df ')
Print Stdout.read ()

Transport.close ()
Copy Code
Connection based on public key:
Example
Import Paramiko

Private_key = Paramiko. Rsakey.from_private_key_file ('/home/auto/.ssh/id_rsa ')

# Create an SSH object
SSH = Paramiko. Sshclient ()
# Allow connections to hosts that are not in the Know_hosts file
Ssh.set_missing_host_key_policy (Paramiko. Autoaddpolicy ())
# Connection Server
Ssh.connect (hostname= ' c1.salt.com ', port=22, username= ' Wupeiqi ', Key=private_key)

# Execute Command
stdin, stdout, stderr = Ssh.exec_command (' df ')
# Get command results
result = Stdout.read ()

# Close Connection
Ssh.close ()

Example
Import Paramiko

Private_key = Paramiko. Rsakey.from_private_key_file ('/home/auto/.ssh/id_rsa ')

Transport = Paramiko. Transport (' hostname ', 22)
Transport.connect (username= ' Wupeiqi ', Pkey=private_key)

SSH = Paramiko. Sshclient ()
Ssh._transport = Transport

stdin, stdout, stderr = Ssh.exec_command (' df ')

Transport.close ()

2.SFTPClient

For connecting to a remote server and performing an upload download

Upload and download based on user name password

Import Paramiko

Transport = Paramiko. Transport (' hostname ', 22)
Transport.connect (username= ' Wupeiqi ', password= ' 123 ')

SFTP = Paramiko. Sftpclient.from_transport (transport)
# upload location.py to the server/tmp/test.py
Sftp.put ('/tmp/location.py ', '/tmp/test.py ')
# download Remove_path to local Local_path
Sftp.get (' Remove_path ', ' Local_path ')

Transport.close ()
Upload and download based on public key keys

Import Paramiko

Private_key = Paramiko. Rsakey.from_private_key_file ('/home/auto/.ssh/id_rsa ')

Transport = Paramiko. Transport (' hostname ', 22)
Transport.connect (username= ' Wupeiqi ', Pkey=private_key)

SFTP = Paramiko. Sftpclient.from_transport (transport)
# upload location.py to the server/tmp/test.py
Sftp.put ('/tmp/location.py ', '/tmp/test.py ')
# download Remove_path to local Local_path
Sftp.get (' Remove_path ', ' Local_path ')

Transport.close ()

#!/usr/bin/env python
#-*-Coding:utf-8-*-
Import Paramiko
Import UUID

Class Haproxy (object):

def __init__ (self):
Self.host = ' 172.16.103.191 '
Self.port = 22
Self.username = ' Wupeiqi '
Self.pwd = ' 123 '
Self.__k = None

def create_file (self):
file_name = str (UUID.UUID4 ())
With open (file_name, ' W ') as F:
F.write (' SB ')
return file_name

def run (self):
Self.connect ()
Self.upload ()
Self.rename ()
Self.close ()

def connect (self):
Transport = Paramiko. Transport ((Self.host,self.port))
Transport.connect (USERNAME=SELF.USERNAME,PASSWORD=SELF.PWD)
Self.__transport = Transport

def close (self):

Self.__transport.close ()

def upload (self):
# Connect, Upload
file_name = Self.create_file ()

SFTP = Paramiko. Sftpclient.from_transport (Self.__transport)
# upload location.py to the server/tmp/test.py
Sftp.put (file_name, '/home/wupeiqi/tttttttttttt.py ')

def rename (self):

SSH = Paramiko. Sshclient ()
Ssh._transport = Self.__transport
# Execute Command
stdin, stdout, stderr = Ssh.exec_command (' mv/home/wupeiqi/tttttttttttt.py/home/wupeiqi/ooooooooo.py ')
# Get command results
result = Stdout.read ()

ha = Haproxy ()
Ha.run ()

Iv. basic operation of MySQL database

1. Database operation

show databases;
use [databasename];
Create database [name];

2. Data table operation

Show tables;

CREATE TABLE Students
(
ID int NOT NULL auto_increment primary key,
Name Char (8) is not NULL,
Sex char (4) NOT NULL,
Age tinyint unsigned is not NULL,
Tel char (+) NULL default "-"
);

CREATE TABLE ' Wb_blog ' (
' ID ' smallint (8) unsigned not NULL,
' CatID ' smallint (5) unsigned not NULL DEFAULT ' 0 ',
' title ' varchar (+) not NULL DEFAULT ' ',
' Content ' text is not NULL,
PRIMARY KEY (' id '),
UNIQUE KEY ' catename ' (' catid ')
) ;

3. Data manipulation

INSERT into students (Name,sex,age,tel) VALUES (' Alex ', ' Mans ', 18, ' 151515151 ')

Delete from students where id = 2;

Update students set name = ' SB ' WHERE id = 1;

SELECT * FROM Students

4. Other

Primary key
FOREIGN key
Left and right connections

Python operations Redis, RabbitMQ, SQLAlchemy, Paramiko, MySQL