Redis Cache Database

Introduction to Caching databases

NoSQL (NoSQL = not-only sql), meaning "not just SQL", refers to non-relational databases, With the rise of internet web2.0 website, the traditional relational database in coping with web2.0 website, especially the web2.0 pure dynamic website of ultra-large-scale and high-concurrency SNS type, has been unable to overcome, exposing a lot of difficult problems, and the non-relational database has been developed very rapidly because of its own characteristics. NoSQL databases are created to address the challenges of multiple data types in large-scale data sets, especially big data application challenges.

Four categories of NoSQL databases

Key value (key-value) Store database

This type of database primarily uses a hash table that has a specific key and a pointer to the specific data. The advantage of the Key/value model for IT systems is simplicity and ease of deployment. But if the DBA only queries or updates part of the value, Key/value becomes inefficient. [3] Examples include: Tokyo cabinet/tyrant, Redis, Voldemort, Oracle BDB. The column stores the database. This part of the database is often used to deal with massive amounts of data for distributed storage. Keys still exist, but they are characterized by pointing to multiple columns. These columns are arranged by the column family. such as: Cassandra, HBase, Riak. Document Type DatabaseThe document database is inspired by Lotus Notes Office software and is similar to the first key-value store. This type of data model is a versioned document, and semi-structured documents are stored in a specific format, such as JSON. A document database can be considered an upgraded version of a key-value database, allowing for the nesting of key values. and the document database is more efficient than the key-value database query. such as: CouchDB, MongoDb. Domestic also has the document type database SEQUOIADB, already open source. Graph Database (graph)The graphical structure of the database is different from the other columns and the rigid structure of the SQL database, it is using a flexible graphical model, and can be extended to multiple servers. NoSQL databases do not have a standard query language (SQL), so database queries require a data model. Many NoSQL databases have rest-type data interfaces or query APIs. [2] such as: neo4j, Infogrid, Infinite Graph. Therefore, we summarize the NoSQL database in the following cases, the comparison is applicable: 1, the data model is relatively simple, 2, the need for more flexible IT systems, 3, the database performance requirements are high; 4, There is no need for a high degree of data consistency; 5. For a given key, it is easier to map a complex value environment. Analysis of four classified forms of NoSQL database

category	Examples Example	Typical application Scenarios	Data Model	Advantages	Disadvantages
Key value (Key-value) [3]	Tokyo cabinet/tyrant, Redis, Voldemort, Oracle BDB	Content caching, which is used primarily for high-access loads that handle large amounts of data, for some log systems, and so on. [3]	Key-value pairs that point to value and are usually implemented with hash table [3]	Fast Search Speed	Data is unstructured and is usually used only as a string or binary data [3]
Columnstore database [3]	Cassandra, HBase, Riak	Distributed File Systems	To store the same column of data in a clustered type	Find Fast, scalable, and easily distributed extensions	function relative limitation
Document type database [3]	CouchDB, MongoDb	Web applications (similar to Key-value, value is structured, but the database is able to understand the contents of value)	Key-value corresponding key-value pairs, value is structured data	Data structure requirements are not strict, table structure is variable, do not need to be like a relational database need to pre-defined table structure	Query performance is not high, and the lack of uniform query syntax.
Graph (graph) database [3]	Neo4j, Infogrid, Infinite Graph	Social networks, referral systems, and more. Focus on building a relationship map	Graph structure	Using graph structure correlation algorithm. such as shortest path addressing, N-degree relationship lookup, etc.	Many times need to calculate the entire graph to get the information needed, and this structure is not very good for the distributed cluster scheme. [3]

About Redis

Redis is one of the industry's leading key-value NoSQL databases. Similar to memcached, it supports storing more value types, including string (string), list (linked list), set (set), Zset (sorted set-ordered collection), and hash (hash type). These data types support Push/pop, Add/remove, and intersection-set and difference sets, and richer operations, and these operations are atomic. Based on this, Redis supports sorting in a variety of different ways. As with memcached, data is cached in memory to ensure efficiency. The difference is that Redis periodically writes the updated data to disk or writes the modified operation to the appended record file, and Master-slave (Master-Slave) synchronization is implemented on this basis.

Redis Benefits

• Unusually fast: Redis is very fast and can perform approximately 110000 setup operations per second, and 81,000 per second read operations.

• Support for rich data types: Redis support most developers already know data types such as lists, collections, sortable collections, hashes, and so on.

  This makes it easy to solve a variety of problems in your application because we know which issues to work with which data types are better addressed.

• Operations are atomic: All Redis operations are atomic, ensuring that when two customers access the Redis server at the same time, the updated values (the latest value) are obtained.

• Multiutility Tools: Redis is a multifunctional utility that can be used in many such as: cache, Message delivery queue (Redis native support publish/subscribe), in applications such as: Web application sessions, website page hits, etc. any short data;

Installing Redis

Yum Install Redis

Start Redis

Redis-server  # 6379

Command line into REDIREDIS-CLI

Python installation Module

pip3.5 Install Redis

The Redis API uses

How to link

1. Single link

Redis-py provides two classes of Redis and Strictredis for implementing Redis commands, Strictredis is used to implement most of the official commands, and using official syntax and commands, Redis is a subclass of Strictredis, Used for backwards compatibility with older versions of Redis-py.

# link Import Redisr = Redis. Redis (host= ' Remoteip ', port=6379) r.set (' name ', ' Joker ') print (R.get (' name '))  # Joker

2. Link Pool

Redis-py uses connection pool to manage all connections to a Redis server, avoiding the overhead of each establishment and release of the connection. By default, each Redis instance maintains its own pool of connections. You can create a connection pool directly, and then as a parameter Redis, you can implement multiple Redis instances to share a single connection pool.

# link Pool Import redispool = Redis. ConnectionPool (host= ' Remoteip ', port=6379) R = Redis. Redis (Connection_pool=pool) r.set (' name ', ' Joker ') print (R.get (' name '))

Operation 1. String manipulation

The string in Redis is stored in memory by a name corresponding to a value

Set (name, value, Ex=none, Px=none, Nx=false, Xx=false)

Set the value in Redis, default, create without, modify parameter if present:     ex, expiry time (seconds)     px, Expiration Time (msec)     NX, if set to true, the current set operation executes XX only if name does not exist.     If set to true, the pre-post set operation is performed only if name exists

SETNX (key value)

Set the value to perform the set operation (add) only if name does not exist

Setex (name, value, time)

# set the value # parameter: # time, expiration (number of seconds or Timedelta object)

Psetex (name, Time_ms, value)

# set the value # parameter: # Time_ms, Expiration time (numeric milliseconds or Timedelta object)

Mset (*args, **kwargs)

Batch setting values such as:    Mset (k1= ' v1 ', k2= ' v2 ')    or    mget ({' K1 ': ' v1 ', ' K2 ': ' V2 '})

Get (name)

Get value

Mget (keys, *args)

Bulk gain such as:    mget (' name ', ' age ')    or    r.mget ([' Name ', ' age '])

Getset (name, value)

Set a new value and get the original value

GetRange (key, start, end)

# Get Subsequence (get from Byte, non-character) # parameter:    # Name,redis name    # Start, start position (bytes)    # end, end position (bytes) # For example: "Li", 0-3 means "Lee"

###

Setbit (name, offset, value)

# operation on binary representation of name corresponding value # parameter:    # Name,redis name    # offset, bit index (convert value to binary and then index)    # value can only be 1 or 0 # Note: If you are in Redis There is a corresponding: N1 = "foo",        then the binary representation of String Foo is: 01100110 01101111 01101111    So, if you execute Setbit (' N1 ', 7, 1), the 7th bit is set to 1,        Then the final binary becomes 01100111 01101111 01101111, namely: "Goo" # extension, conversion binary representation:     # Source = "Wu Jianzi"    Source = "Foo" for     I in source:
   num = Ord (i)        print bin (num). replace (' B ', ')     special, what if the source is a kanji "Wu Jianzi"? A    : For Utf-8, each Chinese character accounts for 3 bytes, then "Wu Jianzi" has 9 bytes       for Chinese characters, the For loop is iterated by byte, then each byte is converted to decimal number at iteration, then the decimal number is converted        to binary 11100110 10101101 10100110 11100110 10110010 10011011 11101001 10111101 10010000        -------------------------------- ----------------------------------------------------,                    Wu-                         pei-                           qi

* Examples of use, in the most space-saving way, the number of online users and which users are online

Getbit (name, offset)

# Gets the value of a bit in the binary representation of the name corresponding to the value (0 or 1)

Bitcount (Key, Start=none, End=none)

# Gets the value of name corresponding to the number of 1 in the binary representation # parameter:    # Key,redis name    # start, bit start position    # end, bit end position

strlen (name)

# returns the byte length of the name corresponding value (3 bytes for a kanji)

INCR (self, name, amount=1)

# self-increment name corresponding to the value, when the name does not exist, then create Name=amount, otherwise, the self-increment. # parameter:    # Name,redis name    # amount, self increment (must be integer) # Note: Same as Incrby

Incrbyfloat (self, name, amount=1.0)

# self-increment name corresponding to the value, when the name does not exist, then create Name=amount, otherwise, the self-increment. # parameters:    # Name,redis name    # amount, self-increment (floating point)

DECR (self, name, amount=1)

# self-Subtract the value of name, when name does not exist, create name=amount, otherwise, subtract. # parameters:    # Name,redis name    # amount, self-meiosis (integer)

Append (key, value)

# append content after Redis name Value # parameter:    key, Redis name    value, string to append

2. Hash operation

Hash expression form Some like Pyhton in the dict, can store a group of strong association of data, Redis in memory storage format as follows

Hset (name, key, value)

# name corresponding to the hash set a key value pair (not present, then create; otherwise, modify) # parameter:    # Name,redis's name    # key,name corresponding hash of key    # value, The value of the hash in name corresponding to # Note:    # HSETNX (name, key, value), created when the current key does not exist in the hash of name (equivalent to adding)

Hmset (name, mapping)

# in the hash name corresponding to the batch set key value pair # parameter:    # Name,redis name    # mapping, dictionary, such as: {' K1 ': ' v1 ', ' K2 ': ' v2 '} # such as:    # r.hmset (' xx ', {' K1 ': ' v1 ', ' K2 ': ' V2 '})

Hget (Name,key)

# Gets the value based on key in the hash of name

Hmget (name, keys, *args)

# Get the value of multiple keys in the hash of name # parameter:    # name,reids corresponds to the name    # keys, to get the key collection, such as: [' K1 ', ' K2 ', ' K3 ']    # *args, to get the key, such as: K1,k2,k3 #    R.mget (' xx ', [' K1 ', ' K2 '])    # or    # print r.hmget (' xx ', ' K1 ', ' K2 ')

Hgetall (name)

# Gets the name of all the key values corresponding to the hash

Hlen (name)

# Gets the number of key-value pairs in the hash of the name

Hkeys (name)

# Gets the value of all keys in the hash of name

Hvals (name)

# Gets the value of all the values in the hash of the name corresponding to

Hexists (name, key)

# Check if the hash name corresponds to the current key being passed in

Hdel (Name,*keys)

# Delete the key value of the specified key in the hash name corresponding to the

Hincrby (name, key, Amount=1)

# self-increment name corresponds to the value of the specified key in the hash, does not exist to create the key=amount# parameter:    # Name,redis in the name    # key, hash corresponding key    # amount, self-increment (whole number)

Hincrbyfloat (name, key, amount=1.0)

# self-increment name corresponds to the value of the specified key in the hash, does not exist to create the Key=amount # parameter:    # Name,redis in the name    # key, hash corresponding key    # amount, self-increment (floating-point number) # Self-increment name corresponds to the value of the specified key in the hash and does not exist to create the Key=amount

Hscan (name, cursor=0, Match=none, Count=none)

Start a full hash scan with:

HSCAN myhash 0

Start a hash scan with the fields matching a pattern with:

HSCAN myhash 0 MATCH order_*

Start a hash scan with fields matching a pattern and forcing the scan command to does more scanning with:

HSCAN myhash 0 MATCH order_* COUNT 1000

# Incremental iterative acquisition, the data is very useful for large numbers, Hscan can achieve the shard of data, not one-time to get all the data, so that the memory is burst # parameter:    # Name,redis's name    # cursor, Cursor (fetch data based on the cursor in batches)    # match, matches the specified key, the default none means all key    # count, the minimum number of shards per shard, and the default none indicates the number of default shards using Redis # such as:    # First time: Cursor1, data1 = R.hscan (' xx ', cursor=0, Match=none, Count=none)    # second time: Cursor2, data1 = R.hscan (' xx ', cursor=curs Or1, Match=none, Count=none)    # ...    # until the value of the cursor in the return value is 0 o'clock, indicating that the data has been fetched through the Shard

Hscan_iter (name, Match=none, Count=none)

# Create generators using yield encapsulation hscan to get data in batches to Redis  # parameters: # match    , matches the specified key, default none means all key    # count, the minimum number of fetches per shard, Default None indicates the number of default shards using Redis  # such as:    # for item in R.hscan_iter (' xx '):    #     Print Item

3. List

List operation, the list in Redis is stored in memory by a name corresponding to a list.

Lpush (name,values)

# add elements to the list of name, and each new element is added to the leftmost # example:    # r.lpush (' oo ', 11,22,33)    # Save order: 33,22,11 # Extension: # Rpush (name, values) table Show right-to-left operation

Lpushx (Name,value)

# Add an element to the list of name, and the value is added to the far left of the listing only if name already exists # more: # RPUSHX (name, value) means right-to-left operation

Llen (name)

# name corresponds to the number of list elements

Linsert (name, where, Refvalue, value))

# Insert a new value before or after a value in the list corresponding to name # parameter:    # Name,redis's name    # Where,before or after    # Refvalue, the benchmark value, that is: Insert data before and after it    # Value, the data to be inserted

R.lset (name, index, value)

# Reassign the value of an index in the list corresponding to name # parameter:    # Name,redis's name    # index,list index position    # value, values to set

R.lrem (name, value, num)

# Delete the specified value in the list of name # parameter:    # Name,redis name    # value, to delete the value    # num,  num=0, delete all the specified values in the list;           # num=2, From front to back, delete 2;           # num=-2, from back to front, delete 2

Lpop (name)

# Gets the first element on the left side of the list of name and removes it from the list, and the return value is the first element # more: # Rpop (name) means right-to-left operation

Lindex (name, index)

# gets list elements in the list of name corresponding to the index

Lrange (name, start, end)

# in name corresponding list shard get data # parameter:    # Name,redis name    # Start, index start position    # end, index end position

LTrim (name, start, end)

# Remove the value from the name corresponding to the Start-end index # parameter:    # Name,redis's name    # Start, index start position    # end, index end position

Rpoplpush (SRC, DST)

# Remove the rightmost element from a list and add it to the far left of the other list # parameter:    # SRC, to fetch the list of data name    # DST, the name of the list to add data to

Blpop (keys, timeout)

# Arrange multiple lists, follow the elements from left to right to go to the pop corresponding list # parameter:    # Keys,redis's name Collection    # timeout, time-out, when elements of all list elements get finished, block the wait list with data in seconds, 0 means to block forever # more:    # r.brpop (keys, timeout), getting data from right to left

Brpoplpush (SRC, DST, timeout=0)

# Remove an element from the right side of a list and add it to the left of the other list # parameter:    # src, remove and remove the list of elements corresponding to the name    # DST, to insert the list of elements corresponding to the name    # timeout, When there is no data in the SRC corresponding list, blocking waits for its data timeout (in seconds), 0 means forever blocking

4.set Collection Operations

Set operation, set set is a list that does not allow duplicates

Sadd (name,values)

# add elements to the collection that corresponds to name

SCard (name)

# Gets the number of elements in the collection for name

Sdiff (keys, *args)

# The collection of elements in the first name corresponding collection and not in the other name corresponding collection

Sdiffstore (dest, Keys, *args)

# Gets the collection that corresponds to the first name and is not in the other name, and then adds it to the corresponding collection of dest

Sinter (keys, *args)

# get the set of one more name corresponding to the collection

Sinterstore (dest, Keys, *args)

# get the set of one more name corresponding to the set, and then add it to the dest corresponding collection

Sismember (name, value)

# Check if value is a member of the collection that corresponds to name

Smembers (name)

# gets all members of the collection that the name corresponds to

Smove (SRC, DST, value)

# move a member from one collection to another collection

Spop (name)

# randomly removes a member from a collection and returns it

Srandmember (name, numbers)

# randomly fetching numbers elements from a collection of name

Srem (name, values)

# Delete some values in the collection of name

Sunion (keys, *args)

# Gets the set of one more name corresponding to the collection

Sunionstore (Dest,keys, *args)

# Gets the set of one more name corresponding to the collection and saves the result to the corresponding collection of dest

Sscan (name, cursor=0, Match=none, Count=none)
Sscan_iter (name, Match=none, Count=none)

# Operations with strings, used for incremental iterations to get elements in batches, avoiding too much memory consumption

An ordered set, on the basis of the set, sorts each element; The ordering of the elements needs to be compared according to another value, so for an ordered set, each element has two values, namely: values and fractions, which are specifically used for sorting.

Other common operations

Delete (*names)

# Depending on the deletion of any data type in Redis

Exists (name)

# detects if a Redis name exists

Keys (pattern= ' * ')

# get Redis's name from model # more:    # KEYS * matches all keys in the database.    # KEYS H?llo matches Hello, Hallo and Hxllo.    # KEYS H*llo matches Hllo and Heeeeello.    # KEYS H[ae]llo matches hello and Hallo, but does not match Hillo

Expire (name, time)

# Set a time-out for one of the Redis's name and the element disappears after timeout

Rename (src, DST)

# Rename the name of Redis to

Move (name, DB))

# move one of the Redis values to the specified db if there is no movement on the DB

Randomkey ()

# randomly get the name of a redis (not deleted)

Type (name)

# Gets the type of the name corresponding value

Scan (cursor=0, Match=none, Count=none)
Scan_iter (Match=none, Count=none)

# same string operation, used for incremental iteration to get key

　　

Pipeline

Redis-py The default is to create each request (Connection pool request connection) and disconnect (return connection pool) One connection operation, if you want to specify more than one command in a single request, you can use Pipline to implement a single request to specify multiple commands, and by default Pipline is an atomic operation.

Import Redis,timepool = Redis. ConnectionPool (host= ' Remoreip ', port=6379) R = Redis. Redis (connection_pool=pool) # pipe = R.pipeline (transaction=false) pipe = R.pipeline (transaction=true) pipe.set (' name ' , ' JOKER ') time.sleep () Pipe.set (' role ', ' ADMIN ') Pipe.execute ()

Publish a subscription

Publisher: Server

Demo is as follows

ImportRedisclassRedishelper:def __init__(self): self.__conn= Redis. Redis (host=' Remoteip') Self.chan_sub='fm104.5'self.chan_pub='fm104.5'    defPublic (Self, msg): Self.__conn. Publish (Self.chan_pub, msg) # Send MessagereturnTruedefSubscribe (self): Pub= self.__conn. PubSub () # Start subscription, turn on Radio Pub.subscribe (SELF.CHAN_SUB) # Tune Channel Pub.parse_response () # Ready to receive, in call It's time to take a look at the subscribers.
        returnPubredis Helper

Subscribers: Dashboad and data processing

#!/usr/bin/env python#-*-coding:utf-8-*-from Monitor. Redishelper import Redishelper obj = Redishelper () redis_sub = Obj.subscribe () while True:    msg= redis_sub.parse_ Response ()  # receives a message and does not receive the card to be Stuck    print (msg)

Published by:

#!/usr/bin/env python#-*-coding:utf-8-*-from Monitor. Redishelper import Redishelper obj = Redishelper () obj.public (' Hello ')

Command line Send Message

Publish fm104.5 Hello

More Versatile Command http://doc.redisfans.com/

Redis Cache Database