Python Nosql-redis connections, pipelines

The difference between a non-relational database and a relational database:

Benefits of non-relational databases:

1. Performance NoSQL is based on key-value pairs, which can be imagined as the corresponding relationship between the primary key and the value in the table, and does not need to be parsed by the SQL layer, so the performance is very high.

2. Scalability is also due to the fact that there is no coupling between data based on key-value pairs, so it is easy to scale horizontally.

The benefits of a relational database:

1. Complex queries can easily use SQL statements to make very complex data queries between a table and multiple tables.

2. Transactional support enables data access requirements that are high for security performance. For these two types of databases, the other's advantage is their own weakness, and vice versa. But in recent years both of these databases have evolved in a different direction.

For example, NoSQL databases are slowly beginning to have some of the complex query functions of SQL database, such as Couchbase's index and MONGO's complex queries. Support for transactions can also use some system-level atomic operations to implement methods such as optimistic locking to curve the salvation.

SQL database also began to evolve slowly, such as the implementation of Handlersocker technology, can be implemented on the SQL layer of MySQL penetration, in a nosql way to access the database, performance can be achieved or even beyond the NoSQL database. scalability, such as Percona Server, enables the implementation of a cluster that is not centralized. Although the Poles are beginning to evolve another pole because of their weaknesses, the increase in these features will also weaken the advantages they would otherwise have, such as the increase in index on the couchbase gradually reduces the read and write performance of the database.

The concept of Redis:

Redis is a key-value storage system.

Similar to memcached, it supports storing more value types, including string (string), list (linked list), set (set), Zset (sorted set-ordered collection), and hash (hash type).

These data types support Push/pop, Add/remove, and intersection-set and difference sets, and richer operations, and these operations are atomic.

Based on this, Redis supports sorting in a variety of different ways. As with memcached, data is cached in memory to ensure efficiency. The difference is that Redis periodically writes the updated data to disk or writes the modified operation to the appended record file, and Master-slave (Master-Slave) synchronization is implemented on this basis.

Redis is a high-performance Key-value database.

The emergence of Redis, to a large extent, compensates for the lack of memcached such key/value storage, in some cases can be a good complement to the relational database. It provides clients such as Java,c/c++,c#,php,javascript,perl,object-c,python,ruby,erlang, which is convenient to use.

Redis supports master-slave synchronization. Data can be synchronized from the primary server to any number of slave servers, from the server to the primary server that is associated with other slave servers.

This enables Redis to perform single-layer tree replication. You can write to the data intentionally or unintentionally. Because of the full implementation of the publish/subscribe mechanism, you can subscribe to a channel and receive a complete message release record from the master server when the tree is synchronized anywhere from the database. Synchronization is helpful for the scalability and data redundancy of read operations.

Installation of Redis:

Redis Chinese web: http://www.redis.net.cn/

[[email protected] opt]# wget http://download.redis.io/releases/redis-3.0.6.tar.gz[[email protected] Opt ]# tar zxvf redis-3.0.6.tar.gz-c/usr/local/src/[[email protected] opt]# CD! $CD/usr/local/src/[[email  Protected] src]# lspip-1.3.1.tar.gz Python-2.7.14 redis-3.0.6[[email protected] src]# CD Redis-3.0.6/[[email  protected] redis-3.0.6]# ls00-releasenotes contributing Deps Makefile README runtest runtest-s Entinel src utilsbugs COPYING INSTALL MANIFESTO redis.conf runtest-cluster sentinel.conf tes Ts[[email protected] redis-3.0.6]# makecd src && make all ... [[email protected] redis-3.0.6]# src/redis-server &[1] 50007[[email protected] redis-3.0.6]# 50007:C 28 Nov 08:53:03.999 # warning:no config file specified, using the default Config. In order to specify a config file use src/redis-server/path/to/redis.conf50007:m 08:53:04.000 * increased maximum Number of open FileS to 10032 (it is originally set to 1024x768). 50007:m 08:53:04.001 # Creating Server TCP listening socket *:6379:bind : Address already in use[1]+ Exit 1 src/redis-server[[email protected] redis-3.0.6]# PS Aux|grep re        Disroot 8345 0.1 0.4 143920 8004? Sl Nov27 1:43/src/redis-server *:6379root 50009 0.0 0.0 103320 884 pts/1 s+ 08:53 0:00 grep redis[[e                   Mail protected] redis-3.0.6]# netstat-nlp|grep redistcp 0 0 0.0.0.0:6379 0.0.0.0:* LISTEN 8345/./src/redis-se TCP 0 0::: 6379:::* LI  STEN 8345/./src/redis-se [[email protected] redis-3.0.6]#

Python Operation Redis

Installing the Redis module

[[email protected] redis-3.0.6]# pip install redisCollecting redis  Using cached redis-2.10.6-py2.py3-none-any.whlInstalling collected packages: redisSuccessfully installed redis-2.10.6

Redis. Redis Connection Database

Set () Method: writes a single piece of data

Get (Key) method: Gets the value corresponding to the key

Key () method: Get all Keys

import redis as redisredis_info = {    ‘host‘:‘11.11.11.11‘,    ‘port‘:6379}r = redis.Redis(**redis_info)      # 建立redis 连接r.set(‘name‘,‘lijunjiang‘)              # r.set()  写入一条数据   redis经key-value形式存储r.get(‘name‘)                             # r.get(key-name)  获取key-name 对应的值print(r.key())                               # r.key 获取所有的键########C:\Python27\python.exe D:/Python/redis/redis_Redis.pylijunjiang[‘name‘]Process finished with exit code 0

Redis. ConnectionPool () Method: Manage all connections to a Redis server to avoid the overhead of each build and release of the connection

By default, each Redis instance maintains its own pool of connections. You can create a connection pool directly, and then as a parameter Redis, you can implement multiple Redis instances to share a single connection pool

import redis as redisredis_info = {    ‘host‘:‘11.11.11.11‘,    ‘port‘:6379}pool = redis.ConnectionPool(**redis_info)   # 建立一个redis 连接池r = redis.Redis(connection_pool=pool)        # 从连接池里获取连接r.set(‘age‘,15)print(r.get(‘age‘))print(r.keys())#######C:\Python27\python.exe D:/Python/redis/redis_Redis.py15[‘age‘, ‘name‘]Process finished with exit code 0

Redis Pipeline

Redis-py The default is to create each request (Connection pool request connection) and disconnect (return connection pool) One connection operation, if you want to specify more than one command in a single request, you can use Pipline to implement a single request to specify multiple commands, and by default Pipline is atomic operation

Redis is a CS-mode TCP server that uses a request-response protocol similar to HTTP. A client can initiate multiple request commands from a single socket connection. After each request command is issued, the client usually blocks and waits for the Redis service to process, and the result is returned to the client via a response message after the Redis process has finished processing the request command

#!/usr/bin/env python#-*-coding:utf-8-*-# @Time: 2017/11/28 22:48# @Author: lijunjiang# @File: redis-pipeline.py Import Datetimeimport Redisredis_info = {' host ': ' 11.11.11.11 ', ' Port ': 6379}def Conncet_redis (): Pool = redis.c Onnectionpool (**redis_info) try:r = Redis. Redis (Connection_pool=pool) except Exception as Err:raise err return rdef withpipe (R): # r = Conncet_redi s () pipe = R.pipeline (transaction=true) for I in Xrange (1,1000): key = "Test1" + str (i) value = "Value "+str (i) Pipe.set (key, value) Pipe.execute () def withoutpipe (R): # r = Conncet_redis () for I in Xrange (1,10 XX): key = "Test1" + str (i) value = "Value" +str (i) R.set (key, value) def time (fun): R = conncet_red    Is () start = Datetime.datetime.now () fun (r) end = Datetime.datetime.now () Run_time = (End-start). microseconds return Run_timeif __name__ = = ' __main__ ': # r1 = Conncet_redis () # r2 = Conncet_rediS () Withpipe_time = time (withpipe) Withoutpipe_time = time (withoutpipe) print ("Withpipe Run time:{0}". Format (with Pipe_time)) Print ("Withoutpipe run Time: {0}". Format (withoutpipe_time)) ################## #C: \python27\python.exe D :/python/redis/redis-pipeline.pywithpipe run Time:19000withoutpipe run time:203000process finished with exit code 0

Python nosql-redis connections, pipelines