Cache Database Redis

What is Redis?

Redis is a TCP server that supports request/response protocols. In Redis, the request is completed using the following steps:

• The client sends a query to the server and reads from the socket, usually in a blocking manner, for the server response.
• The server processes the command and sends the response back to the client.

Redis is an open source (BSD-licensed), in-memory data structure storage system that can be used as a database, cache, and message middleware. It supports multiple types of data structures such as strings (strings), hashes (hashes), lists (lists), collections (sets), ordered collections (sorted sets) and range queries, bitmaps, hyperloglogs and geospatial (GEOSP atial) index RADIUS query. Redis has built-in replication (replication), LUA scripting (LUA scripting), LRU driver events (LRU eviction), transactions (transactions), and different levels of disk persistence (persistence), and Redis Sentinel (Sentinel) and automatic partitioning (Cluster) provide high availability (HI availability).

So the question is, since we're talking about caching database Redis, and just knowing about Redis, what's the relationship between a redis and a cache database?

Cache database

NoSQL (NoSQL = not-only SQL) means more than just a database.

NoSQL, which refers to a non-relational database, With the rise of Internet web2.0 Web site, the traditional relational database to cope with web2.0 website, especially the super large-scale and high-concurrency SNS type web2.0 pure dynamic website has appeared to be inadequate, exposing a lot of insurmountable problems, and non-relational database because of its own characteristics have been very rapid development. NoSQL databases are created to solve the challenges of large-scale collection of multiple data types, especially big data application challenges.

So if someone says that NoSQL is Redis, then it's wrong to say that Redis is just one of the four categories of NoSQL.

Four categories of NoSQL databases

Key value (key-value) store database
This type of database primarily uses a hash table that has a specific key and a pointer to the specific data. The advantage of the Key/value model for IT systems is simplicity and ease of deployment. But if the DBA only queries or updates part of the value, Key/value becomes inefficient. [3] Examples include: Tokyo cabinet/tyrant, Redis, Voldemort, Oracle BDB.
The column stores the database.
This part of the database is often used to deal with massive amounts of data for distributed storage. Keys still exist, but they are characterized by pointing to multiple columns. These columns are arranged by the column family. such as: Cassandra, HBase, Riak.
Document Type Database
The document database is inspired by Lotus Notes Office software and is similar to the first key-value store. This type of data model is a versioned document, and semi-structured documents are stored in a specific format, such as JSON. A document database can be considered an upgraded version of a key-value database, allowing for the nesting of key values. and the document database is more efficient than the key-value database query. such as: CouchDB, MongoDb. Domestic also has the document type database SEQUOIADB, already open source.
Graph Database (graph)
The graphical structure of the database is different from the other columns and the rigid structure of the SQL database, it is using a flexible graphical model, and can be extended to multiple servers. NoSQL databases do not have a standard query language (SQL), so database queries require a data model. Many NoSQL databases have rest-type data interfaces or query APIs. [2] such as: neo4j, Infogrid, Infinite Graph.

Therefore, we summarize the NoSQL database in the following cases where the comparison applies:

• 1, the data model is relatively simple;
• 2, the need for more flexible IT systems;
• 3, the database performance requirements are high;
• 4, do not need a high degree of data consistency;
• 5, for a given key, it is easier to map complex values of the environment.

Classification	Examples Example	Typical application Scenarios	Data model	Advantages	Disadvantages
Key value (Key-value)	Tokyo Cabinet/tyrant, Redis, Voldemort, Oracle BDB	Content caching, Used primarily for high-access workloads that handle large amounts of data, Also used for some log systems and so on.	Key points to Value Key-value pairs, usually using the Hash table to achieve	Fast Search Speed	Data is unstructured, is usually used only as a word. Character string or binary Data
Column Store Database	Cassandra, HBase, Riak	Distributed File Systems	Cluster-based storage, The same column data exists Together	Find Fast, Scalability is strong, Easier to Distributed extensions	function relative limitation
Document Type Database	Couchdb,mongodb	Web Apps (similar to Key-value, Value is structured, and the difference is the number The contents of the value can be understood by the library)	Key-value Correspondence A key-value pair, value As structured data	Data structure requirements Not strict, table structure Variable, does not need to be like relational database One Sample to pre-define the table Structure	Query performance is not high, and lack of unity The query syntax.
Graph (graph) Database	NEO4J, Infogrid, Infinite Graph	Social networks, referral systems, and more. Focus on building a relationship map	Graph structure	Using graph structure Correlation calculation Method. such as the shortest path Addressing, n-degree relationship checking Find and wait	A lot of times it takes a whole A figure to do a calculation to get The information needed, and And this structure is not very good Do distributed cluster-side Case.

Applicable scenarios

• 1, the data model is relatively simple;
• 2, the need for more flexible IT systems;
• 3, the database performance requirements are high;
• 4, do not need a high degree of data consistency;
• 5, for a given key, it is easier to map complex values of the environment.

Redis

Redis is one of the industry's leading key-value NoSQL databases. Similar to memcached, it supports storing more value types, including string (string), list (linked list), set (set), Zset (sorted set-ordered collection), and hash (hash type). These data types support Push/pop, Add/remove, and intersection-set and difference sets, and richer operations, and these operations are atomic. Based on this, Redis supports sorting in a variety of different ways. As with memcached, data is cached in memory to ensure efficiency. The difference is that Redis periodically writes the updated data to disk or writes the modified operation to the appended record file, and Master-slave (Master-Slave) synchronization is implemented on this basis.

Advantages

• Unusually fast: Redis is very fast and can perform approximately 110000 setup operations per second, and 81,000 per second read operations.
• Support for rich data types: Redis support most developers already know data types such as lists, collections, sortable collections, hashes, and so on.
• This makes it easy to solve a variety of problems in your application because we know which issues to work with which data types are better addressed.
• Operations are atomic: All Redis operations are atomic, ensuring that when two customers access the Redis server at the same time, the updated values (the latest value) are obtained.
• Multiutility Tools: Redis is a versatile utility that can be used in many such as: cache, messaging Queues (Redis native support publish/subscribe), in applications such as Web application sessions, website page hits, and any short-term data.

Before use, you need to install the configuration for its environment

Under Ubuntu, you only need to enter Apt-get install Redis-server. After the installation is complete, Redis-server starts the service, and then redis-cli can manipulate it.

We can set a password for it to ensure its security into vi/etc/redis.conf to Requirepass ' password '.

Remote link redis-cli-h 127.0.0.1-p 6379-a "MyPassword"

In a python environment, Pip install Redis is required

If the link on Python does not need to repair the configuration file under the Bind, enter vi/etc/redis.conf under Bind 0.0.0.0 can be changed.

Connection mode, without password, do not fill in password

Import REDISR = Redis. Redis (host= ' 10.211.55.4 ', port=6379,password= ' MyPassword ')

Common operations for Redis

String

The string in Redis is stored in memory by a name corresponding to a value

View the set command what to use

Get (name): Get value

Mget (keys, *args)

Bulk gain such as:    mget (' ylr ', ' Wupeiqi ')    or    r.mget ([' Ylr ', ' Wupeiqi '])

127.0.0.1:6379> help set  set key value [EX seconds] [PX milliseconds] [nx| XX]  Summary:set The string value of a key  since:1.0.0  group:string

Set the value in Redis, default, create without, modify parameter if present:     ex, expiry time (seconds)     px, Expiration Time (msec)     NX, if set to true, the current set operation executes XX only if name does not exist.     If set to true, the pre-post set operation is performed only if name exists

SETNX (name, value)

Set the value to perform the set operation (add) only if name does not exist

Setex (name, Time,value)

# set the value # parameter: # time, expiration (number of seconds or Timedelta object)

Psetex (name, Time_ms, value)

# set the value # parameter: # Time_ms, Expiration time (numeric milliseconds or Timedelta object)

Mset (*args, **kwargs)

Batch setting values such as:    Mset (k1= ' v1 ', k2= ' v2 ')    or    mget ({' K1 ': ' v1 ', ' K2 ': ' V2 '})

Getset (name, value)

Set a new value and get the original value

GetRange (key, start, end)

# Gets the subsequence (obtained by Byte, non-character)    # a man 3 bytes

SETRANGE (name, offset, value)

# Modify string contents, start backward replacement from the specified string index (add back if the new value is too long) # parameter:    # Offset, index of string, byte (three bytes for a kanji)    # Value, value to set

Setbit (name, offset, value)

# operation on binary representation of name corresponding value # parameter:    # Name,redis name    # offset, bit index (transform value to binary and then index)    # value can only be 1 or 0

Getbit (name, offset)

# Gets the value of a bit in the binary representation of the name corresponding to the value (0 or 1)

Bitcount (Key, Start=none, End=none)

# Gets the value of name corresponding to the number of 1 in the binary representation # parameter:    # Key,redis name    # start, bit start position    # end, bit end position

strlen (name)

# returns the byte length of the name corresponding value (3 bytes for a kanji)

INCR (self, name, amount=1)

# self-increment name corresponding to the value, when the name does not exist, then create Name=amount, otherwise, the self-increment. # parameter:    # Name,redis name    # amount, self increment (must be integer) # Note: Same as Incrby

DECR (self, Name, amount=1): auto-Subtract

Incrbyfloat (self, name, amount=1.0)

# self-increment name corresponding to the value, when the name does not exist, then create Name=amount, otherwise, the self-increment. # parameters:    # Name,redis name    # amount, self-increment (floating point)

Append (key, value)

# append content after Redis name Value # parameter:    key, Redis name    value, string to append

Hash operation

Hash form Some like Pyhton in the dict, can store a group of strong association of data, in Redis, each hash (hash) can save up to 400 million key value pairs

Hscan (name, cursor=0, Match=none, Count=none)

# Incremental iterative acquisition, the data is very useful for large numbers, Hscan can achieve the shard of data, not one-time to get all the data, so that the memory is burst # parameter:    # Name,redis's name    # cursor, Cursor (fetch data based on the cursor in batches)    # match, matches the specified key, the default none means all key    # count, the minimum number of shards per shard, and the default none indicates the number of default shards using Redis # such as:    # First time: Cursor1, data1 = R.hscan (' xx ', cursor=0, Match=none, Count=none)    # second time: Cursor2, data1 = R.hscan (' xx ', cursor=curs Or1, Match=none, Count=none)    # ...    # until the value of the cursor in the return value is 0 o'clock, indicating that the data has been fetched through the Shard

Hscan_iter (name, Match=none, Count=none)

# Create generators using yield encapsulation hscan to get data in batches to Redis  # parameters: # match    , matches the specified key, default none means all key    # count, the minimum number of fetches per shard, Default None indicates the default number of shards using Redis  

The common commands are as follows

Command	Explain
Hdel key Field2 [Field2]	Deletes one or more hash fields.
Hexists key Field	Determines whether a hash field exists.
Hget key field	Gets the value of the hash field stored in the specified key.
Hgetall Key	Gets all fields and values stored in the hash of the specified key
Hincrby key field Increment	Increments the integer value of a hash field by a given number
Hincrbyfloat key field Increment	Increments the floating point value of a hash field by a given number
Hkeys Key	Get all the fields in the hash
Hlen Key	Get the number of fields in a hash
Hmget key field1 [Field2]	Gets the value of all given hash fields
Hmset key field1 value1 [Field2 value2]	Set their values for multiple hash fields, respectively
Hset key field value	Set the string value of a hash field
Hsetnx key field value	Set the value of the hash field only if the field does not exist
Hvals Key	Get all values in the hash

　　

　　

　　

　　

　　

　　

　　

　　

　　

　　

List

List operation, the list in Redis is stored as a list in memory according to a name

Lpush (name,values)

# add elements to the list of name, and each new element is added to the left-most # example:    # r.lpush (' oo ', 11,22,33)    # Save order: 33,22,11 # Extension:    # Rpush (name, Val UES) indicates right-to-left operation

The remaining command details are listed below

Command	Explain
Blpop key1 [Key2] timeout	Deletes and gets the first element in the list, or blocks until an element is available
Brpop key1 [Key2] timeout	Deletes and gets the last element in the list, or blocks until an element is available
Brpoplpush Source Destination Timeout	POPs a value from the list, pushes it to another list, and returns it; or blocked until one is available
LINDEX Key index	Get an element from a list by its index
Linsert key Before/after Pivot value	Insert an element before or after another element in the list
Llen Key	Get the length of a list
Lpop Key	Delete and get the first element in a list
Lpush key value1 [value2]	Add one or more values to the list
Lpushx Key value	Add a value to a list only if the list exists
Lrange key start Stop	Get a series of elements from a list
Lrem Key count Value	Remove an element from the list
LSET Key index value	To set the value of an element in a list by index
LTRIM key start Stop	Trim a specified range of lists
Rpop Key	Delete and get the last element in the list
Rpoplpush Source Destination	Deletes the last element in the list, attaches it to another list, and returns
Rpush key value1 [value2]	Attaching one or more values to a list
Rpushx Key value	Append values to the list only if the list exists

Set Set

is to not allow duplicate lists

Sadd (name,values)

# add elements to the collection that corresponds to name

SCard (name)
Gets the number of elements in the collection for name
Sdiff (keys, *args)
in the first name corresponding collection and not in the collection of other name-corresponding collections
Sdiffstore ( Dest, Keys, *args)
Gets the collection of the first name corresponding to the collection that does not correspond to the other name, and then adds it to the dest corresponding collection
Sinter (keys, *args)
Gets the set of one more name corresponding collection 　　Br>sinterstore (dest, Keys, *args)
Gets the set of one more name corresponding collection, and then adds it to the dest corresponding collection
Sismember (name, value)
Checks if value is a member of the collection for name
Smembers (name)
gets all members of the collection that name corresponds to
smove (SRC, DST, value)
to move a member from one collection to another collection
Spop (name)
Removes a member from the Right (trailer) of the collection and returns it
Srandmember (name, numbers)
to randomly get numbers elements from the collection of name
Srem ( Name, values)
Delete certain values in the collection of name
Sunion (keys, *args)
Get the set of multiple name-corresponding sets
Sunionstore (Dest,keys, *args) The
gets the set of one more name-corresponding collection and saves the result to the corresponding collection of Dest
Sscan (name, cursor=0, Match=none, Count=none)
incrementally iterates over the elements in the collection
Sscan_ ITER (name, Match=none, Count=none)
operation with strings for incremental iterations to get elements in batches, avoiding too much memory consumption

An ordered set, on the basis of the set, sorts each element; The ordering of the elements needs to be compared according to another value, so for an ordered set, each element has two values, namely: values and fractions, which are specifically used for sorting .

Zadd (name, *args, **kwargs)

# add elements to the ordered collection of name # such as:     # zadd (' zz ', ' N1 ', 1, ' N2 ', 2)     # or     # zadd (' zz ', n1=11, n2=22)

Zrank (name, value)

Gets the rank of a value in the ordered collection of name (starting at 0) # more:    # Zrevrank (name, value), from large to small sort

The remaining approximate and unordered collection usages are the same.

Other common operations　

Delete (*names)
Depending on the deletion of any data type in Redis
Exists (name)
Detects if a Redis name exists
Keys (pattern= ' * ')
Get the name of Redis based on the model
More:
# KEYS * matches all keys in the database.
# KEYS H?llo matches Hello, Hallo and Hxllo.
# KEYS H*llo matches Hllo and Heeeeello.
# KEYS H[ae]llo matches hello and Hallo, but does not match Hillo
Expire (name, time)
Set a time-out for one of the Redis's name
Rename (src, newsrc)
Rename the name of Redis to
Move (name, DB))
Move one of the Redis values to the specified db
Randomkey ()
Randomly get the name of a redis (not deleted)
Type (name)
Gets the type of the name corresponding value

Redis Pipeline

The basic implication of a pipeline is that the client can send multiple requests to the server without waiting for a reply, and eventually waiting for a reply in a single step.

The use of pipelines in Python is as follows

Because Edis-py is created by default on each request (connection pooling request connection) and disconnects (return connection pool) One connection operation, if you want to specify multiple commands in a single request, you can use Pipline to implement a single request to specify multiple commands, and by default Pipline is an atomic operation.

Import Redispool = Redis. ConnectionPool (host= ' 10.211.55.4 ', port=6379) R = Redis. Redis (connection_pool=pool) # pipe = R.pipeline (transaction=false) pipe = R.pipeline (transaction=true) pipe.set (' name ' , ' AA ') pipe.set (' role ', ' BB ') Pipe.execute ()

The benefits of piping　

Greatly improved protocol performance, through the pipeline from the connection to the local host speed increased by 5 times times, the Internet connection at least 100 times times faster.

Redis Publish Subscriptions

A Redis Publish Subscription (PUB/SUB) is a message communication pattern: the Sender (pub) sends a message and the Subscriber (sub) receives the message.
The Redis Publish Subscription (PUB/SUB) implements the messaging system, where the sender (known as the publisher in Redis terminology) sends a message when the recipient (the Subscriber) receives the message. The link that transmits the message is called the channel.

Implementing the code in Python is as follows

 from Import  = redishelper () obj.public ('hello')

published by

 from Import  == while True:    msg= redis_sub.parse_response ()     Print msg

subscribed by

Cache Database Redis