Redis Cache Database

redis introduction
Redis is one of the mainstream key-value nosql databases in the industry. Similar to Memcached, it supports relatively more stored value types, including string (string), list (list), set (collection), zset (sorted set-ordered set) and hash (hash type). These data types all support push / pop, add / remove, take intersection, union and difference, and richer operations, and these operations are atomic. On this basis, redis supports sorting in various ways. Like memcached, in order to ensure efficiency, data is cached in memory. The difference is that redis periodically writes updated data to disk or writes modification operations to additional log files, and on this basis, master-slave synchronization is achieved.

Redis advantages
Exceptionally fast: Redis is very fast and can perform about 110,000 set operations per second and 81,000 read operations per second.

Support for rich data types: Redis supports the data types that most developers already know, such as lists, collections, sortable collections, and hashes.

This makes it easy to solve various problems in the application, because we know which problems deal with which data types are better solved.
Operations are atomic: All Redis operations are atomic, to ensure that when two clients access the Redis server at the same time, the updated value (the latest value) is obtained.

MultiUtility tool: Redis is a versatile utility that can be used in many such as: cache, messaging queue (Redis natively supports publish / subscribe), in applications such as: web application sessions, website page clicks, etc. Ephemeral data
redis installation
Install under centos:

 wget http://download.redis.io/releases/redis-4.0.9.tar.gz
 tar xvf redis-4.0.9.tar.gz
 cd redis-4.0.9 /
 make && make install
Start: /mnt/redis-4.0.9/src/redis-server /mnt/redis-4.0.9/redis.conf &

Or modify the configuration file redis.conf option daemonize to yes to start the daemon. This redis version defaults to no: /mnt/redis-4.0.9/src/redis-server /mnt/redis-4.0.9/redis.conf

Redis API usage
Redis supports five data types: string (string), hash (hash), list (list), set (set) and zset (sorted set: ordered set).

The usage of redis-py API can be classified as:

Connection method
connection pool
operating
String operation
Hash operation
List operation
Set operation
Sort Set operation
pipeline
Publish and subscribe
redis connection
1. Operation mode

redis-py provides two classes of Redis and StrictRedis to implement Redis commands. StrictRedis is used to implement most official commands and uses official syntax and commands. Redis is a subclass of StrictRedis and is used for backward compatibility with older versions. redis-py.

Python remote operation redis:

import redis
r = redis.Redis (host = ‘192.168.10.128’, port = 6379)
r.set (‘name‘, ‘xiaogang’)
print (r.get (‘name’))
2. Connection pool

Redis-py uses a connection pool to manage all connections to a redis server, avoiding the overhead of establishing and releasing connections every time. By default, each Redis instance maintains its own connection pool. You can directly establish a connection pool, and then as a parameter Redis, so that multiple Redis instances can share a connection pool.

Redis operation string (String)
Related commands of the Redis string data type are used to manage redis string values.

View the syntax format command: help

127.0.0.1:6379> help set

  SET key value [EX seconds] [PX milliseconds] [NX | XX]
  summary: Set the string value of a key
  since: 1.0.0
  group: string
set (key, value, ex = None, px = None, nx = False, xx = False)

Set the value in Redis. By default, it is created if it does not exist, and modified if it exists
parameter:
     ex, expiration time (seconds)
     px, expiration time (ms)
     nx, if set to True, the current set operation will be executed only if the key does not exist
     xx, if set to True, the set operation will be executed before the job only when the key exists
 get key

Get value
setnx key value

Set value, only when name does not exist, perform set operation (add)
 setex key value time

# Settings
# Parameter:
    # time, expiration time (number of seconds or timedelta objects)
 psetex key time_ms value

# Settings
# Parameter:
    # time_ms, expiration time (number of milliseconds or timedelta objects)
 mset * args, ** kwargs

Set values in batches
Such as:
127.0.0.1:6379> mset k1 v1 k2 v2 k3 v3 k4 v4
OK
 mget * args ** kwargs

#Bulk get values
127.0.0.1:6379> mget k1 k2 k3 k4
1) "v1"
2) "v2"
3) "v3"
4) "v4"
 getset key value

#Set new value and return to the original value
127.0.0.1:6379> getset k1 v111111
"v1"
 getrange key start end

# Get subsequence (get according to byte, non-character)
# Parameter:
    # start, start position (bytes)
    # end, end position (bytes)
#Letters and digits occupy one byte, three bytes per Chinese character
# Such as:
127.0.0.1:6379> getrange k1 0 6
"v111111"
 setrange key offset value

# Modify the content of the string and replace it starting from the specified string index.
# Parameter:
    # offset, the index of the string, bytes (one Chinese character three bytes)
    # value, the value to be set

127.0.0.1:6379> set k1 "hello world!"
OK
127.0.0.1:6379> setrange k1 6 "redis"
(integer) 12
127.0.0.1:6379> get k1
"hello redis!"
 setbit key offset value

# Operate the bits of the binary representation of the value corresponding to name
 
# Parameter:
    # name, redis name
    # offset, the index of the bit (the value is converted to binary before indexing)
    # value, the value can only be 1 or 0
 
# Note: If there is a correspondence in Redis: n1 = "foo",
        The binary representation of the string foo by ord (), bin () is: 01100110 01101111 01101111
    So, if setbit (‘n1’, 7, 1) is executed, the 7th bit will be set to 1,
        Then the final binary becomes 01100111 01101111 01101111, that is: "goo"
127.0.0.1:6379> set k1 foo
OK
127.0.0.1:6379>
127.0.0.1:6379> setbit k1 7 1
(integer) 0
127.0.0.1:6379> get k1
"goo"
 

 strlen key

#Return the byte length of the value corresponding to name (3 bytes for a Chinese character)
127.0.0.1:6379> get k1
"goo"
127.0.0.1:6379> strlen k1
(integer) 3
 getbit key offset

# Get the value of a bit in the binary representation of the value corresponding to name (0 or 1)
 bitcount key start end

# Get the number of 1s in the binary representation of the value corresponding to name
# Parameter:
    # key, Redis name
    # start, bit start position
    # end, bit end position
 incr key

# key's numeric value is incremented by 1, when the key does not exist, the value is 1
127.0.0.1:6379> set k1 10
OK
127.0.0.1:6379> incr k1
(integer) 11
127.0.0.1:6379> get k1
"11"

127.0.0.1:6379> incr kk
(integer) 1
127.0.0.1:6379> get kk
"1"
 incrby key incement

Add the specified increment value to the number stored in the key.

If the key does not exist, the value of key will be initialized to 0 first, and then the INCRBY command is executed.

If the value contains the wrong type, or a value of type string cannot be represented as a number, then an error is returned.

The value of this operation is limited to 64-bit (bit) signed digital representation.
127.0.0.1:6379> set k1 10
OK
127.0.0.1:6379> incrby k1 29
(integer) 39
127.0.0.1:6379> get k1
"39"
127.0.0.1:6379> incrby kt 29
(integer) 29
 incrbyfloat key incement

#Add the value stored in key to the increment value of the specified floating point number
127.0.0.1:6379> incrbyfloat k1 29.5
"68.5"
 decr key

# Decrease the value corresponding to name. When name does not exist, create name = amount, otherwise, decrement it.
 
# Parameter:
    # name, Redis name
    # amount, decrement (integer)
127.0.0.1:6379> set k1 10
OK
127.0.0.1:6379> decr k1
(integer) 9
 append key

# Append content after the value corresponding to redis name
# Parameter:
    key, redis name
    value, the string to be appended

127.0.0.1:6379> append k1 "hello"
(integer) 6
127.0.0.1:6379> get k1
"9hello"
 Redis Hash
The hash representation is somewhat like the dict in pyhton, which can store a set of strongly related data. The storage format of Hash in memory in redis is as follows:

HSET key field value HMGET

with

key field [field ...]

The Hset command is used to assign values to the fields in the hash table.
If the hash table does not exist, a new hash table is created and HSET operation is performed.
If the field already exists in the hash table, the old value will be overwritten.

The Hmget command is used to return the value of one or more given fields in the hash table.
If the specified field does not exist in the hash table, then return a nil value.
127.0.0.1:6379> hset info name lisi
(integer) 1
127.0.0.1:6379> hset info age 24
(integer) 1
127.0.0.1:6379> hset info sex male
(integer) 1

127.0.0.1:6379> hmget info name age sex
1) "lisi"
2) "24"
3) "male"
 hexists key field
HDEL key field [field ...]

 The Hexists command is used to check whether the specified field of the hash table exists. If the hash table contains the given field, return 1. If the hash table does not contain the given field, or the key does not exist, return 0.

The Redis Hdel command is used to delete one or more specified fields in the hash table key. Fields that do not exist will be ignored.
127.0.0.1:6379> hexists info name
(integer) 1
127.0.0.1:6379> hdel info name
(integer) 1
127.0.0.1:6379> hget info name
(nil)
 hkeys key

hgetall key

hkeys key Get key values of all hash tables
hgetall key returns all the fields and values in the hash table. In the return value, each field name is followed by the field value, so the length of the return value is twice the size of the hash table.
127.0.0.1:6379> hkeys info
1) "age"
2) "sex"
127.0.0.1:6379> hgetall info
1) "age"
2) "24"
3) "sex"
4) "male"
 hincrby key field increment

HINCRBYFLOAT key field increment

The Hincrby command is used to add the specified increment value to the field value in the hash table.
The increment can also be a negative number, which is equivalent to subtracting the specified field.
If the key of the hash table does not exist, a new hash table is created and the HINCRBY command is executed.
If the specified field does not exist, the value of the field is initialized to 0 before executing the command.
Executing the HINCRBY command on a field storing a string value will cause an error.
The value of this operation is limited to 64-bit (bit) signed digital representation.

The HINCRBYFLOAT command is used to add the specified floating point increment value to the field value in the hash table.
If the specified field does not exist, the value of the field is initialized to 0 before executing the command.
127.0.0.1:6379> hincrby info age 1
(integer) 25
127.0.0.1:6379> hincrbyfloat info age 2.5
"27.5"
 hlen key

hvalue key

hlen key Get the number of key-value pairs
hvalue key Get all the values in the hash table
127.0.0.1:6379> hlen info
(integer) 3
127.0.0.1:6379> hvals info
1) "27.5"
2) "male"
3) "1"
127.0.0.1:
 HMSET key field value [field value ...] Set multiple field-value pairs to the hash table key at the same time.

 HSCAN key cursor [MATCH pattern] [COUNT count] Iterate the key-value pairs in the hash table.


The hmset command is used to set multiple field-value pairs into a hash table at the same time.
This command will overwrite the existing fields in the hash table.
If the hash table does not exist, an empty hash table is created and the HMSET operation is performed
hscan iterates key-value pairs in the hash table
127.0.0.1:6379> hmset class python alex linux laonanhai
OK
127.0.0.1:6379> hscan class 3
1) "0"
2) 1) "python"
   2) "alex"
   3) "linux"
   4) "laonanhai"
 redis list (list)

List operation, the List in redis is stored in memory according to a name corresponding to a List.

The Lpush command inserts one or more values into the head of the list. If the key does not exist, an empty list will be created and the LPUSH operation will be performed. When the key exists but is not a list type, an error is returned.

LPUSH key value [value ...] Insert one or more values into the head of the list.

LRANGE key start stop Get the elements in the specified range of the list.

127.0.0.1:6379> lpush names shaowei xiaogang lisi zhangsan
(integer) 9
127.0.0.1:6379> lrange names 0 -1
1) "zhangsan"
2) "lisi"
3) "xiaogang"
4) "shaowei"
   BLPOP key [key ...] timeout Move out and get the first element of the list. If there is no element in the list, the list will be blocked until the timeout expires or a popup element is found.
   BRPOP key [key ...] moves out and gets the last element of the list. If there is no element in the list, it will block the list until the timeout expires or a pop-up element is found.

127.0.0.1:6379> lrange number 0 -1
1) "five"
2) "four"
3) "three"
4) "two"
5) "one"
127.0.0.1:6379> blpop number 1
1) "number"
2) "five"
127.0.0.1:6379> brpop number 1
1) "number"
2) "one"
127.0.0.1:6379> lrange number 0 -1
1) "four"
2) "three"
3) "two"
 LINDEX key index gets the elements in the list by index.

LLEN key Get the length of the list.

127.0.0.1:6379> lindex number 0
"four"
127.0.0.1:6379> lindex number 1
"three"
127.0.0.1:6379> llen number
(integer) 3
 LINSERT key BEFORE | AFTER pivot value inserts elements before or after the elements of the list.

LPUSHX key value inserts a value into the head of an existing list.

127.0.0.1:6379> linsert names before lisi alex
(integer) 5
127.0.0.1:6379> lrange names 0 -1
1) "alex"
2) "lisi"
3) "xiaogang"
4) "shaowei"
5) "zhangsan"

127.0.0.1:6379> lpushx names xiaoming
(integer) 6
127.0.0.1:6379> lrange names 0 -1
1) "xiaoming"
2) "alex"
3) "lisi"
4) "xiaogang"
5) "shaowei"
6) "zhangsan"
 LREM key count value removes list elements.

LSET key index value Set the value of the list element by index.

127.0.0.1:6379> lrem names 0 shaowei
(integer) 1
127.0.0.1:6379> lrange names 0 -1
1) "xiaoming"
2) "alex"
3) "lisi"
4) "xiaogang"
5) "zhangsan"
127.0.0.1:6379> lset names 0 xiaohong
OK
127.0.0.1:6379> lrange names 0 -1
1) "xiaohong"
2) "xiaohong"
3) "lisi"
4) "xiaogang"
5) "zhangsan"
 LTRIM key start stop trims a list, that is, let the list keep only the elements in the specified interval, and elements that are not in the specified interval will be deleted.

Ltrim trims a list (trim), that is, let the list keep only the elements in the specified interval, and elements that are not in the specified interval will be deleted.

Subscript 0 indicates the first element of the list, 1 indicates the second element of the list, and so on. You can also use negative subscripts, with -1 for the last element of the list, -2 for the penultimate element of the list, and so on.

RPOP key removes and gets the last element of the list.

127.0.0.1:6379> lpush numbers one two three four five
(integer) 5
127.0.0.1:6379> ltrim numbers 1 -2
OK
127.0.0.1:6379> lrange numbers 0 -1
1) "four"
2) "three"
3) "two"
127.0.0.1:6379> rpop numbers
"two"
127.0.0.1:6379> lrange numbers 0 -1
1) "four"
2) "three"
 RPOPLPUSH source destination removes the last element of the list and adds the element to another list and returns.

RPUSHX key value adds value to the existing list.

127.0.0.1:6379> lrange names 0 -1
1) "xiaohong"
2) "xiaohong"
3) "lisi"
4) "xiaogang"
127.0.0.1:6379> lrange numbers 0 -1
1) "four"
2) "three"
127.0.0.1:6379> rpoplpush names numbers
"xiaogang"
127.0.0.1:6379> lrange numbers 0 -1
1) "xiaogang"
2) "four"
3) "three"

127.0.0.1:6379> rpushx numbers one
(integer) 4
127.0.0.1:6379> lrange numbers 0 -1
1) "xiaogang"
2) "four"
3) "three"
4) "one"
 redis set

Redis Set is an unordered collection of type String. Collection members are unique, which means that no duplicate data can appear in the collection.

The collection in Redis is implemented through a hash table, so the complexity of adding, deleting, and searching are all O (1).

The maximum number of members in the set is 232-1 (4294967295, each set can store more than 4 billion members).

SADD key member [member ...] adds one or more value elements to the collection.

scard key Get the number of set elements.

127.0.0.1:6379> sadd name1 xiaoming lisi zhangsan
(integer) 3
127.0.0.1:6379> scard name1
(integer) 3
 SDIFF key [key ...] returns the difference set of all given sets.

SDIFFSTORE destination key [key ...] returns the difference of all given sets and stores it in destination.

127.0.0.1:6379> sdiff name1 name2
1) "xiaoming"
2) "zhangsan"
127.0.0.1:6379> sdiffstore name3 name1 name2
(integer) 2
SINTER key [key ...] returns the intersection of all given sets.

SINTERSTORE destination key [key ...] returns the intersection of all given sets and stores them in destination.

127.0.0.1:6379> sadd num1 one two three four
(integer) 4
127.0.0.1:6379> sadd num2 on e 2 two 32
(integer) 4
127.0.0.1:6379> sinter num1 num2
1) "one"
2) "two"
127.0.0.1:6379> sinterstore num3 num2 num1
(integer) 2
SISMEMBER key member judges whether the member element is a member of the set key

SMEMBERS key returns all members in the collection

127.0.0.1:6379> sismember num1 ‘one’
(integer) 1
127.0.0.1:6379> smembers num1
1) "one"
2) "four"
3) "three"
4) "two"
 

SMOVE source destination member moves the member element from the source collection to the destination collection

SPOP key removes and returns a random element in the collection

127.0.0.1:6379> smembers num2
1) "2"
2) "one"
3) "32"
4) "two"
127.0.0.1:6379> smove num1 num2 ‘four’
(integer) 1
127.0.0.1:6379> smembers num2
1) "2"
2) "one"
3) "32"
4) "two"
5) "four"

127.0.0.1:6379> smembers num2
1) "2"
2) "one"
3) "32"
4) "two"
5) "four"
127.0.0.1:6379> spop num2
"32"

SREM key member [member ...] remove one or more elements in the collection

SRANDMEMBER key [count] returns one or more random numbers in the set

127.0.0.1:6379> srem num2 32 noe
(integer) 0
127.0.0.1:6379> srandmember num2 2
1) "one"
twenty two"

SUNION key [key ...] returns the union of all the given sets

SUNIONSTORE destination key1 [key2] The union of all the given collections is stored in the destination collection

127.0.0.1:6379> smembers num1
1) "one"
2) "three"
3) "two"
127.0.0.1:6379> smembers num2
1) "one"
2) "four"
127.0.0.1:6379> sunion num1 num2
1) "one"
2) "four"
3) "three"
4) "two"
127.0.0.1:6379> sunionstore num4 num2 num1
(integer) 4
127.0.0.1:6379> smembers num4
1) "one"
2) "four"
3) "three"
4) "two"

Redis sorted set
Redis ordered collections, like collections, are also collections of string type elements, and do not allow duplicate members.

The difference is that each element is associated with a double type score. Redis uses the score to sort the members of the set from small to large.

The members of an ordered set are unique, but the score can be repeated.

ZADD key score1 member1 [score2 member2] Add one or more elements to an ordered collection, or update the score of an existing member

ZRANGE key start stop [WITHSCORES] returns the ordered set into the members in the specified interval through the index interval

127.0.0.1:6379> zadd z1 8 jack 9 lisi 3 jerry 2 bitch
(integer) 4
127.0.0.1:6379> zrange z1 0 -1 withscores
1) "bitch"
twenty two"
3) "jerry"
4) "3"
5) "jack"
6) "8"
7) "lisi"
8) "9"
 ZCARD key Get the number of members of an ordered collection

ZCOUNT key min max calculates the number of members of the specified interval score in an ordered set

127.0.0.1:6379> zcard z1
(integer) 4
127.0.0.1:6379> zcount z1 2 8
(integer) 3
 ZINCRBY key increment member in an ordered set adds the increment to the specified member's score, you can pass the negative value increment to make the score minus the corresponding value, such as ZINCRBY key -5 member, is to subtract the score value of the member 5.

When the key does not exist, or the score is not a member of the key, ZINCRBY key increment member is equivalent to ZADD key increment member. When the key is not an ordered set type, an error is returned. The fractional value can be an integer value or a double-precision floating-point number.

ZINTERSTORE destination numkeys key [key ...] calculates the intersection of one or more ordered sets and stores the result set in the new ordered set key

127.0.0.1:6379> zincrby z1 19 bitch
"twenty one"
127.0.0.1:6379> zadd z2 89 ‘tom‘ 77 ‘li lianjei‘ 3 ‘jerry’
(integer) 3
127.0.0.1:6379> zinterstore z3 2 z1 z2
(integer) 1
127.0.0.1:6379> zrange z3 0 -1 withscores
1) "jerry"
2) "6"
 ZLEXCOUNT key min max calculates the number of members in the specified dictionary interval in an ordered set

ZRANGEBYLEX key min max [LIMIT offset count] returns the members of an ordered set through a dictionary interval

127.0.0.1:6379> zlexcount z2 [jerry [tom
(integer) 3
127.0.0.1:6379> zrange z2 0 -1 withscores
1) "jerry"
twenty three"
3) "li lianjei"
4) "77"
5) "tom"
6) "89"
127.0.0.1:6379> zrangebylex z2 [jerry [tom
1) "jerry"
2) "li lianjei"
3) "tom"
 ZRANK key member returns the index of the specified member in an ordered collection

ZREM key member [member ...] remove one or more members of the ordered set

127.0.0.1:6379> zrange z2 0 -1 withscores
1) "jerry"
twenty three"
3) "li lianjei"
4) "77"
5) "tom"
6) "89"
127.0.0.1:6379> zrank z2 ‘li lianjei’
(integer) 1
127.0.0.1:6379> zrank z2 ‘jerry’
(integer) 0
127.0.0.1:6379> zrank z2 ‘tom’
(integer) 2
127.0.0.1:6379> zrem z2 ‘jerry’
(integer) 1
 ZSCORE key member returns the ordered set, the member's score value

127.0.0.1:6379> zscore z2 tom
"89"
Redis another command operation
 DEL key [key ...] delete a key

EXISTS key [key ...] Check if the redis key exists
EXPIRE key seconds set the existence time for a redis key

RENAME key newkey renames the key

randomkey get a key randomly

type key Get the corresponding value type of key

Publish and subscribe
Publisher: Server

Subscribers: Dashboad and data processing

Demo is as follows:

import redis
class RedisHelper:

    def __init __ (self):
        self .__ conn = redis.Redis (host = ‘192.168.10.142’)
        self.chan_sub = ‘fm104.5’
        self.chan_pub = ‘fm104.5’

    def public (self, msg):
        self .__ conn.publish (self.chan_pub, msg)
        return True

    def subscribe (self):
        pub = self .__ conn.pubsub () #Turn on the radio
        pub.subscribe (self.chan_sub) #tune channel
        pub.parse_response () #Ready to accept
        return pub
 subscriber

from redishelper import RedisHelper

obj = RedisHelper ()
redis_sub = obj.subscribe ()

while True:
    msg = redis_sub.parse_response ()
    print (msg)
 announcer

 
from monitor.RedisHelper import RedisHelper
obj = RedisHelper ()
obj.public (‘hello‘)
 

Redis cache database