Memcache, Redis

Memcached

Memcached is a high-performance distributed memory object caching system for dynamic Web applications to mitigate database load by caching data and objects in memory to reduce the number of times a database is read, thus increasing the speed of dynamic, database-driven Web sites Memcached based on a storage key/ The hashmap of the value pair, whose daemon (daemon) is written in C, but the client can be written in any language and communicates with the daemon through the Memcached protocol

• Start memcached

  Memcached-d-M 10-u root-l 10.211.55.4-p 12000-c 256-p/tmp/memcached.pid
  Parameter description:
  -D is to start a daemon
  -M is the amount of memory allocated to Memcache, in megabytes
  -U is the user running Memcache
  -L is the server IP address of the listener
  -P is the port that sets Memcache listening, preferably a port above 1024
  The-c option is the maximum number of concurrent connections to run, which is 1024 by default and is set according to the load on your server
  -P is a PID file that is set to save Memcache

• Add to

  Set creates a key-value pair, if it does not exist, is created, otherwise modified
  Set_multi Create multiple key-value pairs, if not present, create, otherwise modify
  Add a key value pair, if there is the same key, error
  Append modify the value of the specified key to add content after the value corresponding to the specified key
  Prepend modifies the value of the specified key, adding content before the value corresponding to the specified key

  mc = memcache.Client([‘127.0.0.1:11211‘,],debug=True)mc.set(‘k1‘,‘v1‘) # k1=v1mc.set_multi({‘k2‘:‘v2‘,‘k3‘:‘v3‘}) # k1=v1,k2=v2mc.add(‘k4‘,‘v4‘) # k4=v4mc.append(‘k1‘,‘after‘) # k1=v1aftermc.prepend(‘k2‘,‘before‘) # k2=beforev2

• Delete

  Delte Delete the specified key-value pair
  Delete_multi Delete multiple key-value pairs

  mc = memcache.Client([‘127.0.0.1:11211‘,],debug=True)mc.delete(‘k1‘)mc.delete_multi([‘k2‘,‘k3‘])

• Modify

  Replace to modify the value of the specified key, if the key does not exist, then the error
  INCR will specify the value of the key corresponding to increment n, if the specified n,n is 1
  decr the value corresponding to the specified key is reduced by N, if the specified n , N for 1
  CAs modifies the value corresponding to the specified key to avoid generating dirty data

    mc = memcache. Client ([' 127.0.0.1:11211 ',],debug=true) mc.replace (' K1 ', ' 123 ') # K1=123mc.replace (' K2 ', ' 123 ') # K2=123mc.replace ('
   K3 ', ' 123 ') # K2=123MC.INCR (' K1 ', 2) # K1=125MC.DECR (' K2 ', 2) # K2=121mc.cas (' K3 ', ' 456 ') # k3=456  

• Inquire

  Get gets the value corresponding to the specified key
  Get_multi get multiple built-in values
  Gets the value corresponding to the specified key to avoid generating dirty data

  mc = memcache.Client([‘127.0.0.1:11211‘,],debug=True)print(mc.get(‘k1‘)) # 125print(mc.get_multi([‘k1‘,‘k2‘,‘k3‘])) # {‘k1‘: ‘125‘, ‘k2‘: ‘121‘, ‘k3‘: ‘456‘}print(mc.gets(‘k1‘)) # 125

  Redis

  Redis is a key-value storage system. Similar to memcached, it supports storing more value types, including string, list, set, Zset, and hash, which support Push/pop, Add/remove, and intersection sets and differences, and richer operations. And these operations are atomic, on this basis, Redis supports a variety of different ways of sorting, as with memcached, in order to ensure efficiency, data are cached in memory, the difference is that Redis will periodically write the updated data to disk or modify the operation to write to the appended record file, The Master-slave (master-Slave) synchronization is implemented on this basis.

• Start Redis

  Start the service side
  Src/redis-server
  Start the client
  Src/redis-cli

• Connection pool

  Redis-py uses connection pool to manage all connections to a Redis server, avoiding the overhead of each build and release, by default, each Redis instance maintains its own pool of connections, can directly establish a connection pool, and then as a parameter Redis To enable multiple Redis instances to share a single connection pool

• String operations in Redis are stored in memory by a name corresponding to a value

  • Set (name, value, Ex=none, Px=none, Nx=false, xx=false) setting value, ex, expiration Time (seconds) px, expiration Time (milliseconds) NX is ture, only name does not exist, current operation is performed, XX is true, Only name exists, current operation is executed

  • SETNX (name, value) sets the value, only if the name does not exist, the operation is performed

  • Setex (name, value, time) sets the value, in seconds, for expiration

  • Psetex (name, Time_ms, value) sets the value, Time_ms indicates the expiration time (in milliseconds)

  • Mset (*args, **kwargs) bulk set values

  • Get (name) get value

  • Mget (keys, *args) bulk acquisition

  • Getset (name, value) sets the new value and gets the old value

  • GetRange (key, start, end) Gets the subsequence, start represents the start position (in bytes), and end indicates the ending position (in bytes)

  • SETRANGE (name, offset, value) modifies the contents of the string, starting from the specified string index and replacing it backwards

  • Setbit (name, offset, value) modifies the string, offset represents the byte index, and value can only be set to 1 or 0

  • Getbit (name, offset) Gets the value of a bit in the binary representation of the value corresponding to the name

  • Bitcount (Key, Start=none, End=none) Gets the value of name corresponding to the number of 1 in the binary representation, start is the starting position, end is the ending position

  • Bitop (operation, dest, keys) gets multiple values and takes a bitwise operation to save the final result to the new name corresponding to the value, Operation,and (and), or (or), not (non), XOR (XOR) dest, New name,key, to find the name

  • strlen (name) returns the byte length of name

  • INCR (self, name, amount=1) increment the value corresponding to name, create Name=amount (integer) When name does not exist

  • Incrbyfloat (self, name, amount=1.0) increment the value corresponding to name, create Name=amount (floating-point number) when name does not exist

  • DECR (self, name, amount=1) decrement the value of name, and create Name=amount (integer) When name does not exist

  • Append (key, value) appends content after the value corresponding to name

    pool = redis.ConnectionPool(host=‘127.0.0.1‘,port=6379)r = redis.Redis(connection_pool=pool)r.set(‘k1‘,‘v1‘,ex=None,px=None,nx=False,xx=False)r.setnx(‘k2‘,‘k2‘)r.setex(‘k3‘,‘v3‘,time=1)r.psetex(‘k4‘,time_ms=1000,value=‘v4‘)r.mset({‘k5‘:‘v5‘,‘k6‘:‘v6‘})print(r.get(‘k1‘))print(r.mget([‘k2‘,‘k3‘,‘k4‘]))print(r.getset(‘k5‘,‘V5‘))print(r.getrange(‘k1‘,start=0,end=1))r.setrange(‘k2‘,offset=0,value=1)r.setbit(‘k3‘,offset=1,value=1)print(r.getbit(‘k4‘,offset=1))print(r.bitcount(‘k5‘,start=0,end=-1))r.bitop(‘AND‘,‘k6‘,‘k1‘,‘k2‘)print(r.strlen(‘k1‘))r.incr(‘k2‘, amount=1)r.incrbyfloat( ‘k2‘, amount=1.0)r.decr(‘k3‘,amount=1)r.append(‘k4‘, ‘after‘)

• Hash operation the list in Redis is stored in memory by a name corresponding to a dict

  • Hset (name, key, value) name corresponds to a hash in which a key-value pair is set (not present, then created; otherwise, modified)

  • Hmset (name, mapping) sets the key-value pair in bulk in the hash of name

  • Hget (Name,key) Gets the value based on key in the hash of name

  • Hmget (name, keys, *args) Gets the value of multiple keys in the hash of name

  • Hgetall (name) gets the name of all the key values corresponding to the hash

  • Hlen (name) gets the number of key-value pairs corresponding to the name of the hash

  • Hkeys (name) Gets the value of all keys in the hash corresponding to name

  • Hvals (name) Gets the value of all the values in the hash corresponding to name

  • Hexists (name, key) checks if the hash name corresponds to the current key being passed in

  • Hdel (Name,*keys) deletes the key value of the specified key in the hash name corresponding to

  • Hincrby (name, key, amount=1) self-increment name corresponds to the value of the specified key in the hash, and does not exist to create the Key=amount

  • Hincrbyfloat (name, key, amount=1.0) self-increment name corresponds to the value of the specified key in the hash, and does not exist to create the Key=amount

  • Hscan (name, cursor=0, Match=none, Count=none) incremental iterative acquisition, for the data is very useful, hscan can achieve the Shard of data acquisition, not a one-time to complete the data, so that the memory is burst

  • Hscan_iter (name, Match=none, Count=none) uses yield encapsulation hscan to create generators to get data in batches for Redis

    pool = redis.ConnectionPool(host=‘127.0.0.1‘,port=6379)r = redis.Redis(connection_pool=pool)r.hset(‘hash‘, ‘k1‘, ‘v1‘)r.hmset(‘hash‘,{‘k2‘:‘v2‘,‘k3‘:‘v3‘})print(r.hget(‘bash‘,‘k1‘))print(r.hmget(‘hash‘,‘k1‘,‘k2‘,‘k3‘))print(r.hgetall(‘bash‘))print(r.hlen(‘bash‘))print(r.hkeys(‘bash‘))print(r.hvals(‘bash‘))print(r.hexists(‘bash‘,‘k1‘))print(r.hdel(‘bash‘,‘k1‘,‘k2‘,‘k3‘))r.hincrby(‘bash‘, ‘k1‘, amount=1)r.hincrbyfloat(‘bash‘, ‘k2‘, amount=1.0)print(r.hscan(‘bash‘, cursor=0, match=None, count=None))print(r.hscan_iter(‘bash‘, match=None, count=None))

• List in Redis is stored by a list in memory according to a name

  • Lpush (name,values) adds elements to the list of name, and each new element is added to the leftmost

  • Lpushx (Name,value) adds an element to the list of name, and the value is added to the leftmost side of the listing only if name already exists

  • Llen (name) name corresponds to the number of list elements

  • Linsert (name, where, Refvalue, value) inserts a new value before or after a value in the list corresponding to name

  • LSet (name, index, value) assigns a value to one of the index positions in the list corresponding to name

  • Lrem (name, value, num) deletes the specified value in the list corresponding to name

  • Lpop (name) gets the first element on the left side of the list corresponding to name and removes it from the list, and the return value is the first element

  • Lindex (name, index) gets the list element in the list of name corresponding to the index

  • Lrange (name, start, end) Gets the data in the list of name corresponding to the Shard

  • LTrim (name, start, end) removes the value that is not in the Start-end index in the list corresponding to name

  • Rpoplpush (SRC, DST) remove the rightmost element from a list and add it to the leftmost side of the other list

  • Blpop (keys, timeout) arranges multiple lists, according to the elements of the pop corresponding list from left to right

  • Brpoplpush (SRC, DST, timeout=0) removes an element from the right side of a list and adds it to the left of another list

    pool = redis.ConnectionPool(host=‘127.0.0.1‘,port=6379)r = redis.Redis(connection_pool=pool)r.lpush(‘list‘,11,22,33)r.lpush(‘rlist‘,44,55,66)r.lpushx(‘list‘,44)print(r.llen(‘list‘))r.linsert(‘list‘, ‘after‘, 33, 44)r.lset(‘list‘,0 , 111)r.lrem(‘list‘, 22, 0)print(r.lpop(‘list‘))print(r.lindex(‘list‘, 0))print(r.lrange(‘list‘, 0, -1))print(r.ltrim(‘list‘, 0, -1))r.rpoplpush(‘list‘, ‘rlist‘)r.blpop([‘list‘,‘rlist‘], timeout=1)r.brpoplpush(‘list‘, ‘rlist‘, timeout=0)def list_iter(name): """ 自定义redis列表增量迭代 :param name: redis中的name，即：迭代name对应的列表 :return: yield 返回 列表元素 """ list_count = r.llen(name) for index in range(list_count):     yield r.lindex(name, index)for item in list_iter(‘list‘): print(item)

• Set operation set set is a list that does not allow duplicates

  • Sadd (name,values) name the corresponding collection in which the element is added

  • SCard (name) gets the number of elements in the collection that the name corresponds to

  • Sdiff (keys, *args) The collection of elements in the first name corresponding collection and not in the other name corresponding collection

  • Sdiffstore (dest, keys, *args) Gets the collection that corresponds to the first name and is not in the other name, and then adds it to the dest corresponding collection

  • Sinter (keys, *args) gets the set of one more name corresponding to the set

  • Sinterstore (dest, keys, *args) Gets the set of more than one name corresponding to the set, and then adds it to the dest corresponding collection

  • Sismember (name, value) checks if value is a member of the collection that corresponds to name

  • Smembers (name) gets all members of the collection that the name corresponds to

  • Smove (SRC, DST, value) move a member from one collection to another collection

  • Spop (name) removes a member from the Right (trailer) of the collection and returns it

  • Srandmember (name, numbers) gets numbers elements randomly from the collection that corresponds to name

  • Srem (name, values) removes certain values from the collection that corresponds to name

  • Sunion (keys, *args) gets the set of one more name corresponding to the collection

  • Sunionstore (Dest,keys, *args) gets the set of one more name corresponding to the collection and saves the result to the corresponding collection of dest

  • Sscan (name, cursor=0, Match=none, Count=none) the same string operation, used for incremental iterations to get elements in batches, avoiding too much memory consumption

  • Sscan_iter (name, Match=none, Count=none) the same string operation, used for incremental iterations to fetch elements in batches, avoiding too much memory consumption

    pool = redis.ConnectionPool(host=‘127.0.0.1‘,port=6379)r = redis.Redis(connection_pool=pool)r.sadd(‘set1‘,‘111,222,333‘)r.sadd(‘set2‘,‘222,333,444‘)r.sadd(‘set3‘,‘333,444,555‘)r.sadd(‘set4‘,‘444,555,666‘)print(r.scard(‘set‘))r.sdiff(‘set1‘,‘set2‘,‘set3‘)r.sdiffstore(dest=‘set4‘,keys=[‘set1‘,‘set2‘,‘set3‘])r.sinter(‘set1‘,‘ste2‘,‘set3‘)r.sinterstore(dest=‘set4‘,keys=[‘set1‘,‘set2‘,‘set3‘])r.sismember(‘set1‘,‘111‘)r.smembers(‘set1‘)r.smove(‘set1‘, ‘set2‘, ‘111‘)r.spop(‘set1‘)r.srandmember(‘set1‘, 1)r.srem(‘set1‘, ‘111‘)r.sunion(‘set1‘,‘ste2‘,‘set3‘)r.sunionstore(dest=‘set4‘,keys=[‘set1‘,‘set2‘,‘set3‘])r.sscan(‘set1‘, cursor=0, match=None, count=None)r.sscan_iter(‘set1‘, match=None, count=None)

  Ordered collection

  • Zadd (name, *args, **kwargs) adds an element to the ordered collection of name

  • Zcard (name) Gets the number of ordered collection elements for name

  • Zcount (name, Min, max) gets the number of points in the ordered collection of name corresponding to [Min,max]

  • Zincrby (name, value, amount) the name corresponding to the ordinal collection of the name corresponding to the score

  • R.zrange (name, start, end, Desc=false, Withscores=false, score_cast_func=float) Gets the element of the ordered collection of name corresponding to the index range

  • Zrank (name, value) gets the rank of a value in the ordered collection corresponding to name (starting at 0)

  • Zrangebylex (name, Min, Max, Start=none, Num=none) when all members of an ordered collection have the same score, the elements of an ordered collection are sorted according to the values of the members (lexicographical ordering). This command can return the given ordered set key key, the value of the element is between Min and Max members, each member in the collection is byte by bit (Byte-by-byte Compare), and returns the sorted collection member in order from low to high, If a portion of the two strings is the same, then the command will assume that the longer string is larger than the shorter string.

  • Zrem (name, values) deletes a member of values in the ordered collection corresponding to name

  • Zremrangebyrank (name, Min, max) is deleted according to the line range

  • Zremrangebyscore (name, Min, max) is deleted based on the score range

  • Zremrangebylex (name, Min, max) returns delete based on value

  • Zscore (name, value) gets the fraction of the name corresponding to the value in the Ordered collection

  • Zinterstore (dest, keys, Aggregate=none) gets the intersection of two ordered sets, and if different fractions of the same value are encountered, follow the aggregate

  • Zunionstore (dest, keys, Aggregate=none) Gets the set of two ordered sets, and if different fractions of the same value are encountered, follow the aggregate

  • Zscan (name, cursor=0, Match=none, Count=none, score_cast_func=float) are similar to strings, with the addition of a string to the score_cast_func to manipulate fractions

  • The

  • Zscan_iter (name, Match=none, Count=none,score_cast_func=float) is similar to a string, and is used to manipulate fractions compared to a new score_cast_func of strings

      pool = Redis. ConnectionPool (host= ' 127.0.0.1 ', port=6379) R = Redis. Redis (Connection_pool=pool) r.zadd (' Set ', n1=11,n2=22,n3=33) r.zcard (' Set ') R.zcount (' Set ', one, one) R.zincrby (' Set ', One, 1) r.zrange (' Set ', 0,-1, Desc=false, Withscores=false, score_cast_func=float) R.zrank (' Set ', one) R.zrangebylex (' Set ' ', 11,33,0,2) r.zrem (' Set1 ', [' N1 ', ' N2 ']) R.zremrangebyrank (' Set ', Min, max) R.zremrangebyscore (' Set ', Min, max) R.zremrangebylex (' Set ', Min, max) R.zscore (' Set ', one) r.zinterstore (' Set1 ', [' N1 ', ' N2 '], Aggregate=none) r.zunionstore (' Set1 ', [' N1 ', ' N2 '], Aggregate=none) r.zscan (' Set ', cursor=0, Match=none, Count=none, score_cast_func=float) r.zscan_ ITER (' Set ', Match=none, count=none,score_cast_func=float)  

• Pipeline redis-py is created by default in each request (connection pooling request connection) and disconnected (return connection pool) One connection operation, if you want to specify more than one command in a single request, you can use Pipline to implement a request to specify multiple commands, and by default Pipline is atomic operation

  import redispool = redis.ConnectionPool(host=‘10.211.55.4‘, port=6379)r = redis.Redis(connection_pool=pool)pipe = r.pipeline(transaction=False)pipe = r.pipeline(transaction=True)pipe.multi()pipe.set(‘name‘, ‘kernel‘)pipe.set(‘score‘, ‘good‘) pipe.execute()

• Publish a subscription

  发布者：from monitor.RedisHelper import RedisHelperobj = RedisHelper()obj.public(‘hello‘)订阅者：from monitor.RedisHelper import RedisHelperobj = RedisHelper()redis_sub = obj.subscribe()while True:  msg = redis_sub.parse_response()  print(msg)

• Sentinel

  Redis Heavy Sentinel is primarily used in Redis master-slave replication, and automatically replaces slave with master if master

  from redis.sentinel import Sentinel# 连接哨兵服务器(主机名也可以用域名)sentinel = Sentinel([(‘10.211.55.20‘, 26379),                   (‘10.211.55.20‘, 26380),                   ],                  socket_timeout=0.5)# 获取主服务器地址master = sentinel.discover_master(‘mymaster‘)print(master)# 获取从服务器地址slave = sentinel.discover_slaves(‘mymaster‘)print(slave)# 获取主服务器进行写入master = sentinel.master_for(‘mymaster‘)master.set(‘foo‘, ‘bar‘)# 获取从服务器进行读取（默认是round-roubin）slave = sentinel.slave_for(‘mymaster‘, password=‘redis_auth_pass‘)r_ret = slave.get(‘foo‘)print(r_ret)

Memcache, Redis