Tenth week of Python automated development learning----Redis

Redis

Redis is one of the industry's leading key-value NoSQL databases. Similar to memcached, it supports storing more value types, including string (string), list (linked list), set (set), Zset (sorted set-ordered collection), and hash (hash type). These data types support Push/pop, Add/remove, and intersection-set and difference sets, and richer operations, and these operations are atomic. Based on this, Redis supports sorting in a variety of different ways. As with memcached, data is cached in memory to ensure efficiency. The difference is that Redis periodically writes the updated data to disk or writes the modified operation to the appended record file, and Master-slave (Master-Slave) synchronization is implemented on this basis.

Redis Benefits

• Unusually fast: Redis is very fast and can perform approximately 110000 setup operations per second, and 81,000 per second read operations.

• Support for rich data types: Redis support most developers already know data types such as lists, collections, sortable collections, hashes, and so on.

  This makes it easy to solve a variety of problems in your application because we know which issues to work with which data types are better addressed.

• Operations are atomic: All Redis operations are atomic, ensuring that when two customers access the Redis server at the same time, the updated values (the latest value) are obtained.

• Multiutility Tools: Redis is a multifunctional utility that can be used in many such as: cache, Message delivery queue (Redis native support publish/subscribe), in applications such as: Web application sessions, website page hits, etc. any short data;

Installing the REDIS environment to install Redis on Ubuntu, open the terminal and enter the following command:

$ sudo apt-get update
$ sudo apt-get install redis-server
This will install Redis on your computer
Start Redis

$ redis-server
See if redis is still running

$ redis-cli
This will open a Redis prompt as shown:
redis 127.0.0.1:6379>
In the prompt message above: 127.0.0.1 is the IP address of the machine, and 6379 is the port where the Redis server is running. Now enter the PING command as shown:
redis 127.0.0.1:6379> ping
PONG
This means that you have now successfully installed Redis on your computer. Python operation Redis
sudo pip install redis
or
sudo easy_install redis
or
Source installation
  
See: https://github.com/WoLpH/redis-py
Install Redis Desktop Manager on Ubuntu
To install Redis Desktop Management on Ubuntu, you can download the package from http://redisdesktop.com/download and install it.

Redis Desktop Manager will give you a user interface to manage Redis keys and data. Redis API use connection mode 1, operation mode
redis-py provides two classes of Redis and StrictRedis to implement Redis commands. StrictRedis is used to implement most official commands and uses official syntax and commands. Redis is a subclass of StrictRedis and is used for backward compatibility with older versions. redis-py.

import redis
  
r = redis.Redis (host = ‘10 .211.55.4 ’, port = 6379)
r.set (‘foo‘, ‘Bar’)
print r.get (‘foo‘)
2. Connection pool
Redis-py uses a connection pool to manage all connections to a redis server, avoiding the overhead of establishing and releasing connections every time. By default, each Redis instance maintains its own connection pool. You can directly establish a connection pool, and then as a parameter Redis, so that multiple Redis instances can share a connection pool.

import redis

pool = redis.ConnectionPool (host = ‘10 .211.55.4 ’, port = 6379)
  
r = redis.Redis (connection_pool = pool)
r.set (‘foo‘, ‘Bar’)
print (r.get (‘foo‘))

 
Operation 1. String operation
The String in redis is stored in memory according to a name corresponding to a value.

Set (name, value, ex = None, px = None, nx = False, xx = False)

Set the value in Redis. By default, it is created if it does not exist, and modified if it exists
parameter:
     ex, expiration time (seconds)
     px, expiration time (ms)
     nx, if set to True, the current set operation will only be executed if name does not exist
     xx, if set to True, the set operation will be executed before the post only when name exists
Setnx (name, value)

Set value, only when name does not exist, perform set operation (add)
Setex (name, value, time)

# Settings
# Parameter:
    # time, expiration time (number of seconds or timedelta objects)
Psetex (name, time_ms, value)

# Settings
# Parameter:
    # time_ms, expiration time (number of milliseconds or timedelta objects)
Mset (* args, ** kwargs)

Set values in batches
Such as:
    mset (k1 = ‘v1‘, k2 = ‘v2’)
    or
    mget ({‘k1‘: ‘v1’, ‘k2’: ‘v2’})
Get (name)

Get value
Mget (keys, * args)

Bulk acquisition
Such as:
    mget (‘ylr’, ‘wupeiqi’)
    or
    r.mget ([‘ylr’, ‘wupeiqi’])
Getset (name, value)

Set new value and get original value
Getrange (key, start, end)

# Get subsequence (get according to byte, non-character)
# Parameter:
    # name, the name of Redis
    # start, start position (bytes)
    # end, end position (bytes)
# For example: "Wu Peiqi", 0-3 means "Wu"
Setrange (name, offset, value)

# Modify the content of the string and replace it starting from the specified string index.
# Parameter:
    # offset, the index of the string, bytes (one Chinese character three bytes)
    # value, the value to be set
Setbit (name, offset, value)

# Operate the bits of the binary representation of the value corresponding to name
 
# Parameter:
    # name, redis name
    # offset, the index of the bit (the value is converted to binary before indexing)
    # value, the value can only be 1 or 0
 
# Note: If there is a correspondence in Redis: n1 = "foo",
        Then the binary representation of the string foo is: 01100110 01101111 01101111
    So, if setbit (‘n1’, 7, 1) is executed, the 7th bit will be set to 1,
        Then the final binary becomes 01100111 01101111 01101111, that is: "goo"
 
# Expand, convert binary representation:
 
    # source = "Wu Peiqi"
    source = "foo"
 
    for i in source:
        num = ord (i)
        print bin (num) .replace (‘b‘, ‘‘)
 
    In particular, what if source is the Chinese character "Wu Peiqi"?
    Answer: For UTF-8, each Chinese character occupies 3 bytes, then "Wu Peiqi" has 9 bytes
       For Chinese characters, the for loop will iterate according to bytes, then during iteration, each byte will be converted to a decimal number, and then the decimal number will be converted to binary
        11100110 10101101 10100110 11100110 10110010 10011011 11101001 10111101 10010000
        -------------------------- ------------------------ ----- -----------------------------
                    Wu Pei Qi
* Use example, use the most space-saving way to store the number of online users and which users are online

Getbit (name, offset)

# Get the value of a bit in the binary representation of the value corresponding to name (0 or 1)
Bitcount (key, start = None, end = None)

# Get the number of 1s in the binary representation of the value corresponding to name
# Parameter:
    # key, Redis name
    # start, bit start position
    # end, bit end position
Strlen (name)

# Return the byte length of the value corresponding to name (a Chinese character is 3 bytes)
Incr (self, name, amount = 1)

# Increase the value corresponding to name. When name does not exist, create name = amount, otherwise, increase automatically.
 
# Parameter:
    # name, Redis name
    # amount, self-increasing number (must be an integer)
 
# Note: same as incrby
Incrbyfloat (self, name, amount = 1.0)

# Increase the value corresponding to name. When name does not exist, create name = amount, otherwise, increase automatically.
 
# Parameter:
    # name, Redis name
    # amount, self-increasing number (floating point)
Decr (self, name, amount = 1)

# Decrease the value corresponding to name. When name does not exist, create name = amount, otherwise, decrement it.
 
# Parameter:
    # name, Redis name
    # amount, decrement (integer)
Append (key, value)

# Append content after the value corresponding to redis name
 
# Parameter:
    key, redis name
    value, the string to be appended
2. Hash operation
The form of hash is somewhat like dict in pyhton, which can store a set of strongly related data. The storage format of Hash in memory in redis is as follows:

 Hset (name, key, value)

# Set a key-value pair in the hash corresponding to name (do not exist, create it; otherwise, modify it)
 
# Parameter:
    # name, redis name
    # key, the key in the hash corresponding to name
    # value, the value in the hash corresponding to name
 
# Note:
    # hsetnx (name, key, value), created when the current key does not exist in the hash corresponding to name (equivalent to adding)
Hmset (name, mapping)

# Batch set key-value pairs in the hash corresponding to name
 
# Parameter:
    # name, redis name
    # mapping, dictionary, such as: {‘k1‘: ‘v1’, ‘k2’: ‘v2’}
 
# Such as:
    # r.hmset (‘xx‘, {‘k1‘: ‘v1‘, ‘k2’: ‘v2’})
Hget (name, key)

# Get the value corresponding to the key in the hash corresponding to the name
Hmget (name, keys, * args)

# Get the value of multiple keys in the hash corresponding to name
 
# Parameter:
    # name, the name corresponding to reids
    # keys, to get the key set, such as: [‘k1‘, ‘k2’, ‘k3’]
    # * args, the key to be obtained, such as: k1, k2, k3
 
# Such as:
    # r.mget (‘xx‘, [‘k1‘, ‘k2‘])
    # Or
    # print r.hmget (‘xx‘, ‘k1‘, ‘k2‘)
Hgetall (name)

Get all key values of hash corresponding to name
Hlen (name)

# Get the number of key-value pairs in the hash corresponding to name
Hkeys (name)

# Get the value of all keys in the hash corresponding to name
Hvals (name)

# Get all value values in the hash corresponding to name
Hexists (name, key)

# Check whether the hash corresponding to name has the current incoming key
Hdel (name, * keys)

# Delete the key-value pair of the specified key in the hash corresponding to name
Hincrby (name, key, amount = 1)

# Increase the value of the specified key in the hash corresponding to name, if it does not exist, create key = amount
# Parameter:
    # name, name in redis
    # key, the key corresponding to hash
    # amount, self-increasing number (integer)
Hincrbyfloat (name, key, amount = 1.0)

# Increase the value of the specified key in the hash corresponding to name, if it does not exist, create key = amount
 
# Parameter:
    # name, name in redis
    # key, the key corresponding to hash
    # amount, self-increasing number (floating point)
 
# Increase the value of the specified key in the hash corresponding to name, if it does not exist, create key = amount
Hscan (name, cursor = 0, match = None, count = None)

# Incremental iterative acquisition, very useful for data with large data, hscan can achieve fragmented data acquisition, not all data is acquired at one time, so placing memory is burst
 
# Parameter:
    # name, redis name
    # cursor, cursor (fetch data in batches based on the cursor)
    # match, match the specified key, default None means all keys
    # count, the minimum number of shards for each shard, the default None indicates the number of shards using Redis
 
# Such as:
    # The first time: cursor1, data1 = r.hscan (‘xx‘, cursor = 0, match = None, count = None)
    # The second time: cursor2, data1 = r.hscan (‘xx‘, cursor = cursor1, match = None, count = None)
    # ...
    # Until the return value cursor value is 0, it means that the data has been obtained through sharding
Hscan_iter (name, match = None, count = None)

# Use yield to encapsulate hscan to create a generator, to obtain data in redis in batches
  
# Parameter:
    # match, match the specified key, default None means all keys
    # count, the minimum number of shards for each shard, the default None indicates the number of shards using Redis
  
# Such as:
    # for item in r.hscan_iter (‘xx‘):
    # print item
 
3. List operation
List operation, the List in redis is stored in memory according to a name corresponding to a List.

Lpush (name, values)

# Add elements to the list corresponding to name, each new element is added to the leftmost of the list
 
# Such as:
    # r.lpush (‘oo‘, 11,22,33)
    # The storage order is: 33,22,11
 
# Extension:
    # rpush (name, values) means to operate from right to left
Lpushx (name, value)

# Add elements to the list corresponding to name. Only when name already exists, the value is added to the left of the list
 
# More:
    # rpushx (name, value) means to operate from right to left
Llen (name)

# The number of list elements corresponding to name
Linsert (name, where, refvalue, value))

# Insert a new value before or after a value in the list corresponding to name
 
# Parameter:
    # name, redis name
    # where, BEFORE or AFTER
    # refvalue, the benchmark value, that is: insert data before and after it
    # value, the data to be inserted
R.lset (name, index, value)

# Re-assign a certain index position in the list corresponding to name
 
# Parameter:
    # name, redis name
    # index, the index position of the list
    # value, the value to be set
R.lrem (name, value, num)

# Delete the specified value in the list corresponding to name
 
# Parameter:
    # name, redis name
    # value, the value to be deleted
    # num, num = 0, delete all specified values in the list;
           # num = 2, delete 2 from front to back;
           # num = -2, delete 2 from back to front
Lpop (name)

# Get the first element on the left side of the list corresponding to name and remove it from the list, the return value is the first element
 
# More:
    # rpop (name) means to operate from right to left
Lindex (name, index)

Get list elements according to index in the list corresponding to name
Lrange (name, start, end)

# Get data in the list fragment corresponding to name
# Parameter:
    # name, redis name
    # start, the starting position of the index
    # end, the end of the index
Ltrim (name, start, end)

# Remove the value that is not between the start-end index in the list corresponding to name
# Parameter:
    # name, redis name
    # start, the starting position of the index
    # end, the end of the index
Rpoplpush (src, dst)

# Take the rightmost element from one list and add it to the leftmost of another list
# Parameter:
    # src, the name of the list of data to be fetched
    # dst, the name of the list to add data
Blpop (keys, timeout)

# Arrange multiple lists, pop the elements of the corresponding list from left to right
 
# Parameter:
    # keys, a collection of redis names
    # timeout, timeout time, when all the elements of the list of elements are obtained, the time (seconds) to block waiting for data in the list, 0 means block forever
 
# More:
    # r.brpop (keys, timeout), get data from right to left
Brpoplpush (src, dst, timeout = 0)

# Remove an element from the right side of one list and add it to the left side of another list
 
# Parameter:
    # src, the name corresponding to the list of elements to be removed and removed
    # dst, the name corresponding to the list of elements to be inserted
    # timeout, when there is no data in the list corresponding to src, the timeout time (seconds) for blocking to wait for its data, 0 means blocking forever


4.set set operation
Set operation, Set collection is not allowed to repeat the list

Sadd (name, values)

# Add elements to the collection corresponding to name
Scard (name)

The
Get the number of elements in the set corresponding to name
Sdiff (keys, * args)

The set of elements in the set corresponding to the first name and not in the set corresponding to the other name
Sdiffstore (dest, keys, * args)

# Get the set corresponding to the first name and not the set corresponding to other name, and then add it to the set corresponding to dest
Sinter (keys, * args)

# Get the union of one more set corresponding to name
Sinterstore (dest, keys, * args)

# Get the intersection of one more set corresponding to name, and then add it to the set corresponding to dest
Sismember (name, value)

# Check if value is a member of the set corresponding to name
Smembers (name)

The
# Get all members of the set corresponding to name
Move (src, dst, value)

# Move a member from one collection to another collection
Spop (name)

# Remove a member from the right side (tail) of the collection and return it
Srandmember (name, numbers)

# Randomly get numbers elements from the set corresponding to name
Srem (name, values)

# Delete some values in the set corresponding to name
Sunion (keys, * args)

# Get the union of one more set corresponding to name
Sunionstore (dest, keys, * args)

# Get the union of one more set corresponding to name and save the result to the set corresponding to dest


sscan (name, cursor = 0, match = None, count = None)
sscan_iter (name, match = None, count = None)

# The operation of the same string, used for incremental iteration to obtain elements in batches, to avoid too much memory consumption
 An ordered set, on the basis of the set, sorts each element; the order of elements needs to be compared according to another value, so for an ordered set, each element has two values, namely: value and score Used for sorting.

zadd (name, * args, ** kwargs)

# Add elements to the ordered set corresponding to name
# Such as:
     # zadd (‘zz’, ‘n1‘, 1, ‘n2‘, 2)
     # Or
     # zadd (‘zz’, n1 = 11, n2 = 22)
Zcard (name)

# Get the number of ordered set elements corresponding to name
Zcount (name, min, max)

# Get the number of scores in [min, max] in the ordered set corresponding to name
Zincrby (name, value, amount)

# Increase the score corresponding to the name of the ordered set corresponding to the name
R.zrange (name, start, end, desc = False, withscores = False, score_cast_func = float)

# Get the elements of the ordered set corresponding to name according to the index range
 
# Parameter:
    # name, redis name
    # start, the starting position of the ordered set index (non-score)
    # end, the end position of the ordered set index (non-score)
    # desc, sorting rules, sorting by small to large by default
    # withscores, whether to get the score of the element, by default only the value of the element
    # score_cast_func, a function for data conversion of scores
 
# More:
    # Sort from largest to smallest
    # zrevrange (name, start, end, withscores = False, score_cast_func = float)
 
    # Get the elements of the ordered set corresponding to name according to the score range
    # zrangebyscore (name, min, max, start = None, num = None, withscores = False, score_cast_func = float)
    # Sort from largest to smallest
    # zrevrangebyscore (name, max, min, start = None, num = None, withscores = False, score_cast_func = float)
Zrank (name, value)

# Get the ranking of a value in the ordered set corresponding to name (starting from 0)
 
# More:
    # zrevrank (name, value), sort from largest to smallest
Zrem (name, values)

# Delete the members whose values are values in the ordered set corresponding to name
 
# Such as: zrem (‘zz’, [‘s1‘, ‘s2‘])
Zremrangebyrank (name, min, max)

# Delete according to ranking range
Zremrangebyscore (name, min, max)

# Delete based on score range
Zscore (name, value)

# Get the score corresponding to value in the ordered set corresponding to name
Zinterstore (dest, keys, aggregate = None)

# Obtain the intersection of two ordered sets, if you encounter different scores with the same value, then operate according to aggregate
# The value of aggregate is: SUM MIN MAX
Zunionstore (dest, keys, aggregate = None)

# Get the union of two ordered sets, if you encounter different scores with the same value, then operate according to aggregate
# The value of aggregate is: SUM MIN MAX
Zscan (name, cursor = 0, match = None, count = None, score_cast_func = float)
zscan_iter (name, match = None, count = None, score_cast_func = float)

# Similar to the string, score_cast_func is added to the string to operate on the score


5. Other common operations
delete (* names)

# According to delete any data type in redis
Exists (name)

# Check if the name of redis exists
Keys (pattern = ‘*‘)

# According to the model to get the name of redis
 
# More:
    # KEYS * matches all keys in the database.
    # KEYS h? Llo matches hello, hallo and hxllo etc.
    # KEYS h * llo matches hllo, heeeeello, etc.
    # KEYS h [ae] llo matches hello and hallo but not hillo
Expire (name, time)

# Set a timeout for a name in a redis
Rename (src, dst)

# Rename redis to
Move (name, db))

# Move a value of redis to the specified db
Randomkey ()

# Randomly obtain a redis name (do not delete)
Type (name)

# Get the type of the value corresponding to name
Scan (cursor = 0, match = None, count = None)
scan_iter (match = None, count = None)

# Same string operation, used for incremental iteration to obtain key


pipeline
By default, redis-py will create (connect the pool to apply for a connection) and disconnect (return the connection pool) once for each request. If you want to specify multiple commands in a request, you can use pipline to specify more than one request Commands, and by default a pipline is an atomic operation.

#! / usr / bin / env python
#-*-coding: utf-8-*-
 
import redis
 
pool = redis.ConnectionPool (host = ‘10 .211.55.4 ’, port = 6379)
 
r = redis.Redis (connection_pool = pool)
 
# pipe = r.pipeline (transaction = False)
pipe = r.pipeline (transaction = True)
 
pipe.set (‘name‘, ‘alex’)
pipe.set (‘role’, ‘sb‘)
 
pipe.execute ()
Publish and subscribe
Publisher: Server

Subscribers: Dashboad and data processing

Demo is as follows:

import redis


class RedisHelper:

    def __init __ (self):
        self .__ conn = redis.Redis (host = ‘10 .211.55.4 ’)
        self.chan_sub = ‘fm104.5’
        self.chan_pub = ‘fm104.5’

    def public (self, msg):
        self .__ conn.publish (self.chan_pub, msg)
        return True

    def subscribe (self):
        pub = self .__ conn.pubsub ()
        pub.subscribe (self.chan_sub)
        pub.parse_response ()
        return pub

redis helper
 subscriber:

#! / usr / bin / env python
#-*-coding: utf-8-*-
 
from monitor.RedisHelper import RedisHelper
 
obj = RedisHelper ()
redis_sub = obj.subscribe ()
 
while True:
    msg = redis_sub.parse_response ()
    print msg
announcer:

#! / usr / bin / env python
#-*-coding: utf-8-*-
 
from monitor.RedisHelper import RedisHelper
 
obj = RedisHelper ()
obj.public (‘hello‘)
 

For more information: https://github.com/andymccurdy/redis-py/

http://doc.redisfans.com/

 

 

The tenth week of Python automated development learning-Redis