Redis Usage Scenarios

One. Redis pioneered a new approach to data storage, using Redis, instead of focusing on the problem of how to put an elephant into a refrigerator when faced with a monotonous database, we use Redis's flexible data structure and data manipulation to build different refrigerators for different elephants.

Redis Common data types

The most commonly used data types of Redis are the following five kinds:

String
Hash
List
Set
Sorted Set

Let's start with the analysis of the use of these five types of data and how to implement them internally:

String
Common commands:

Set,get,decr,incr,mget and so on.

Application Scenarios:

String is the most commonly used type of data, and ordinary key/value storage can be categorized as such, which is not explained here.

Implementation method:

String in the Redis internal storage By default is a string, referenced by Redisobject, when encountered INCR,DECR and other operations will be converted to a numeric type for calculation, at this time Redisobject encoding field is an int.

Hash
Common commands:

Hget,hset,hgetall and so on.

Application Scenarios:

Let's simply cite an example to describe the application scenario for a hash, such as storing a user information object data that contains the following information:

The user ID is the key to find, the stored value user object contains the name, age, birthday and other information, if the ordinary key/value structure to store, mainly has the following 2 kinds of storage methods:

The disadvantage of using the user ID as a lookup key to encapsulate other information as a serialized object is to increase the cost of serialization/deserialization and to retrieve the entire object when one of the information needs to be modified, and the modification operation requires concurrency protection. Introduce complex problems such as CAs.

The second method is how many members of this user information object will be saved into the number of key-value, with the user id+ the name of the corresponding property as a unique identifier to obtain the value of the corresponding property, although the cost of serialization and concurrency is omitted, but the user ID is repeated storage, if there is a large number of such data, The memory waste is still very considerable.

So the hash provided by Redis is a good solution to this problem, and the Redis hash is actually the internal stored value as a hashmap, and provides a direct access to the map member's interface, such as:

That is, the key is still the user ID, value is a map, the map key is a member of the property name, value is the property value, so that the data can be modified and accessed directly through its internal map key (Redis called internal map key field), This means that the corresponding attribute data can be manipulated by key (user ID) + field (attribute tag), without the need to store the data repeatedly and without the problem of serialization and concurrency modification control. A good solution to the problem.

It is also important to note that Redis provides an interface (Hgetall) that can fetch all of the property data directly, but if the internal map has a large number of members, it involves traversing the entire internal map, which can be time-consuming due to the Redis single-threaded model. The other client requests are not responding at all, which requires extra attention.

Implementation method:

The above has been said that the Redis hash corresponds to value inside the actual is a hashmap, actually there will be 2 different implementations, this hash of the members of the relatively small redis in order to save memory will be similar to a one-dimensional array to compact storage, without the use of a real HASHMAP structure , the encoding of the corresponding value Redisobject is Zipmap, and when the number of members increases, it automatically turns into a true hashmap, at which time encoding is HT.

List
Common commands:

Lpush,rpush,lpop,rpop,lrange and so on.

Application Scenarios:

Redis list has many applications and is one of the most important data structures of redis, such as Twitter's watchlist, fan list, etc., which can be implemented using the REDIS list structure, which is better understood and not repeated here.

Implementation method:

The implementation of Redis list is a doubly linked list, which can support reverse lookup and traversal, but it is more convenient to operate, but it brings some additional memory overhead, and many implementations within Redis, including sending buffer queues, are also used in this data structure.

Set
Common commands:

Sadd,spop,smembers,sunion and so on.

Application Scenarios:

The functionality provided by Redis set externally is a list-like feature, except that set is automatically weight-saving, and set is a good choice when you need to store a list of data and you don't want duplicate data. and set provides an important interface to determine whether a member is within a set set, which is not available in list.

Implementation method:

The internal implementation of set is a value that is always null hashmap, which is actually calculated by hashing the way to fast weight, which is also set to provide a judge whether a member is within the cause of the collection.

Sorted Set

Common commands:

Zadd,zrange,zrem,zcard, etc.

Usage scenarios:

The usage scenario for Redis sorted set is similar to set, except that the set is not automatically ordered, and the sorted set can be ordered by the user with an additional priority (score) parameter, and is inserted in an orderly, automatic sort. When you need an ordered and non-repeating collection list, you can choose sorted set data structures, such as the public Timeline of Twitter, which can be stored as score in the publication time, which is automatically sorted by time.

Implementation method:

Redis sorted set internal use HashMap and jump Table (skiplist) to ensure the storage and ordering of data, HashMap in the member to score mapping, and the jumping table is all the members, sorted by HashMap in the score , the use of the structure of the jumping table can obtain a relatively high efficiency of finding, and it is relatively simple to implement.

Http://www.cnblogs.com/shanyou/archive/2012/09/04/2670972.html

Two. Sina Weibo: The largest redis cluster in history

Tape is Dead,disk is Tape,flash are Disk,ram Locality is King. -jim Gray

Redis is not a more mature alternative to memcache or MySQL, and is a great addition to architecture for large-scale Internet applications. Now more and more applications are also based on Redis architecture transformation. Let's start by simply announcing the actual Redis platform:

2200+ billion Commands/day 500 billion read/day 50 billion write/day
18tb+ Memory
Servers in 6 IDC 2000+instances
It should be a large Redis use platform at home and abroad, and today we talk about the Redis service platform mainly from the perspective of application.

Pinterest:reids on the maintenance of billions of correlations

Pinterest has become one of the most crazy stories in Silicon Valley, and in 2012, their PC-based business increased by 1047% and mobile adoption increased by 1698%, and the number of independent visits soared to 53.3 billion in March. In Pinterest, people pay attention to tens of millions of things-each user interface queries a board or whether the user is concerned about behavior that contributes to an unusually complex engineering problem. This also gives Redis a niche to play for. After several years of development, Pinterest has become a leader in the media, social and other fields, and its brilliant record is as follows:

Get more recommended traffic than the sum of Google +, YouTube, and LinkedIn
Become the three most popular social networks with Facebook and Twitter
Refer to Pinterest to purchase more users than other sites (more details)
As you would expect, Pinterest's high scale has contributed to a very high demand for IT infrastructure based on its number of independent visits.

Caching to optimize the user experience

Recently, Pinterest engineering manager Abhi Khune to share the user experience needs of the company and the experience of using Redis. Even a breeding application creator will not understand these features before analyzing the details of the site, so take a general look at the usage scenario: first, the pre-check that is mentioned for each fan, and secondly, the UI will accurately display the user's fan and watchlist paging. Performing these operations efficiently requires a very high performance architecture for every click.

Not mundane, Pinterest's software engineers and architects have used MySQL and memcache, but the cache solution still reaches their bottleneck, so for a better user experience, the cache must be expanded. In practice, the engineering team has found that caching only works if the user sub-graph is already in the cache. So. Anyone using this system needs to be cached, which results in a cache of the entire graph. At the same time, the most common answer to whether user A is concerned with User B is often negative, but it is lost as a cache, which results in a database query, so they need a new way to extend the cache. Eventually, their team decided to use Redis to store the entire graph to serve a multitude of lists.

Use Redis to store a large number of Pinterest lists

Pinterest uses Redis as a solution and pushes performance to the in-memory database hierarchy, saving multiple types of lists for users:

fan list
Follow your board user list

followers and non-followers of each board
redis stores all of the above lists for its 70 million users, essentially storing all of the fan graphs, sharding by user ID. Since you can view the data in the above list by type, the analysis profile is stored and accessed using a system that looks more like a transaction. Pinterest the current user like is limited to 100,000, at the beginning of the statistics: if each user is concerned about 25 board, will generate 1.75 billion relationship between users and board. It is also more important that these relationships increase every day as the system is used.

Pinterest's Reids Architecture and operations

One of Pinterest's founders learned that Pinterest began using Python and custom-made Django to write applications that lasted until it had 18 million user-level 410TB user data. While the data is stored using multiple stores, the agent uses 8,192 virtual shards based on the user ID, each running on a Redis db, and 1 Redis instances running multiple Redis db. For full use of the CPU core, multithreading and single-threaded Redis instances are used simultaneously on the same host.

Since the entire data set is running in memory, Redis will persist on Amazon EBS for incoming writes per second. The extension is mainly carried out in two areas: first, to maintain 50% utilization, through the master-slave conversion, half of the Redis instances running on the machine will be translated to a new machine; second, expand nodes and shards. The entire Redis cluster will use a master-slave configuration, from which the part will be treated as a hot backup. Once the primary node fails, the master conversion is completed immediately from the section, and a new slave section will be added, and zookeeper will complete the process. At the same time they run Bgsave on Amazon S3 for more durable storage every hour-this reids is done on the back end, and Pinterest uses that data for mapreduce and analysis jobs. (See the original for more details)

Three different areas of the world to share the experience of Redis combat and use scenes, HTTP://WWW.CSDN.NET/ARTICLE/2013-10-07/2817107-THREE-GIANT-SHARE-REDIS-EXPERIENCE/1

Three. One of the great benefits of Redis is that it is possible to use databases such as MySQL instead of the entire database, but only in some specific scenarios to improve efficiency through Redis's features. This article lists 11 such web scenarios, such as displaying the latest list of items, deleting and filtering, leaderboards, and other related requirements.

Here are 11 web scenarios where you can take advantage of Redis's features to improve efficiency.

1. Displays the most recent list of items in the home page.

Redis uses a cache of resident memory, which is very fast. Lpush is used to insert a content ID that is stored as a keyword in the list header. LTrim is used to limit the maximum number of items in a list to 5000. If the amount of data that the user needs to retrieve exceeds this cache capacity, then the request needs to be sent to the database.

2. Delete and filter.

If an article is deleted, you can use Lrem to completely erase it from the cache.

3. Leaderboards and related issues.

Leaderboards (leader board) are sorted by score. The Zadd command can implement this function directly, and the Zrevrange command can be used to get the top 100 users according to the score, Zrank can be used to get the user rank, very straightforward and easy to operate.

4. Sort by user vote and time.

This is like Reddit's leaderboard, where scores change over time. The Lpush and LTrim commands are combined to add the article to a list. A background task is used to get the list and recalculate the sorting of the list, and the Zadd command is used to populate the build list in the new order. Lists can be retrieved very quickly, even on heavily loaded sites.

5. Expired item processing.

Use Unix time as a keyword to keep the list sorted by time. Search for Current_time and time_to_live to complete the arduous task of finding overdue items. Another background task uses the Zrange ... Withscores the query to delete the expired entries.

6. Count.

The use of various data statistics is very extensive, such as to know when to block an IP address. The Incrby command makes it easy to keep count by atomic increment, Getset to reset the counter, and the Expiration property to confirm when a keyword should be deleted.

7. Specific items within a specific time period.

This is a problem for a particular visitor and can be resolved by using the Sadd command for each page view. Sadd does not add a member that already exists to a collection.

8. Real-time analysis of what is happening, for data statistics and prevention of spam.

Using the Redis Primitives command makes it easier to implement spam filtering systems or other real-time tracking systems.

9. Pub/sub.

Maintaining user mapping of data in an update is a common task in the system. Redis's pub/sub feature uses the subscribe, unsubscribe, and publish commands to make this easier.

10. Queue.

Queues are ubiquitous in the current programming. In addition to the push and pop type commands, Redis also has commands to block queues, allowing one program to be added to the queue by another program at execution time. You can also do something more interesting, such as an RSS feed queue that spins updates.

11. Cache.

The Redis cache is used in the same way as memcache.

Web applications cannot endlessly model wars, look at these redis primitive commands, even though they are simple but powerful and combine them to make it even more unimaginable. Of course, you can write code specifically to do all of this, but Redis is obviously easier to implement.

Http://os.51cto.com/art/201107/278292.htm

Four. 5 common usage scenarios for Redis

In this article, we will describe the most common usage scenarios for Redis, and the different features that affect our choices.

1. Session Cache

One of the most common scenarios for using Redis is the session cache. The advantage of using a Redis cache session over other storage (such as memcached) is that Redis provides persistence. When maintaining a cache that is not strictly consistent, if the user's shopping cart information is lost, most people will be unhappy, now, they will do this?

Fortunately, with Redis's improvements over the years, it's easy to find out how to properly use Redis to cache your session's documentation. Even the well-known business platform Magento offers Redis plugins.

2, full-page cache (FPC)

In addition to basic session tokens, Redis offers a simple FPC platform. Back to the consistency issue, even though the Redis instance was restarted, because of the persistence of the disk, the user will not see a drop in page loading speed, which is a great improvement, similar to PHP native FPC.

Again, for example Magento, Magento provides a plug-in to use Redis as the full-page cache backend.

In addition, for WordPress users, Pantheon has a very good plug-in Wp-redis, this plugin can help you to load the fastest speed you have visited the page.

3. Queue

One of the great advantages of Reids in the memory storage engine area is the provision of list and set operations, which enables Redis to be used as a good Message Queuing platform. The operation that Redis uses as a queue is similar to the push/pop operation of a local program language (such as Python) on a list.

If you quickly search for "Redis queues" in Google, you'll soon be able to find lots of open source projects that are designed to use Redis to create very good back-end tools to meet a variety of queue requirements. For example, celery has a background where redis is used as a broker, which you can view from here.

4. Leaderboard/Counter

Redis's operation of incrementing or decreasing numbers in memory is very well implemented. The set (set) and the ordered set (Sorted set) also make it very simple to perform these operations, and Redis just provides the two data structures exactly. So, we're going to get the top 10 users from the sorted collection – What we call "User_scores", we just need to do it like this:

Of course, it is assumed that you are doing an incremental sort based on your user's score. If you want to return users and users ' scores, you need to do this:

Zrange User_scores 0 Withscores

Agora Games is a good example of what you can see in Ruby, which uses Redis to store data.

5. Publish/Subscribe

Last (but certainly not least) is the Publish/Subscribe feature of Redis. There are really a lot of usage scenarios for publish/subscribe. I've seen people use it in social networking connections, as a publish/subscribe-based scripting trigger, and even use Redis's publish/Subscribe feature to build chat systems! (No, it's true, you can check it out).

All of the features that Redis offers, I feel this is the least of the people who like it, although it provides users with this versatility.

Wait a minute?

Dear reader, What do you think of it? What are you going to do with Redis?

http://blog.jobbole.com/88383/

Five. Redis authors talk about Redis application scenarios

Without a doubt, Redis pioneered a new approach to data storage, and using Redis, we don't have to focus on the problem of how to put an elephant into a refrigerator in the face of a monotonous database, but instead use Redis's flexible data structure and data manipulation to build different refrigerators for different elephants. I hope you like this metaphor.
Here is a fresh article, the author of the Redis author @antirez, describing some of the applications that Redis is more suitable for, Nosqlfan simply listed here for you to list:

1. Take the latest n data Operation
For example, the most recent article to get your site, by the following way, we can put the latest 5,000 reviews of the ID in the Redis list collection, and will be out of the collection part from the database
Use the Lpush latest.comments<id> command to insert data into the list collection
Insert complete and then use the LTrim latest.comments 0 5000 command to keep the last 5,000 IDs forever
then we can use the following logic (pseudo code) when we get a page comment on the client.
FUNCTION get_latest_comments (start,num_items):
id_list = Redis.lrange ("latest.comments", start,start+num_items-1)
IF Id_list.length < Num_items
id_list = sql_db ("Select ... ORDER by Time LIMIT ... ")
END
RETURN id_list
END
If you have a different filter dimension, such as the newest n for a category, you can build a list that is categorized by this category, and Redis is very efficient if you save the ID.

2. Leaderboard application, Top n operation
This requirement differs from the above requirements in that the preceding operation takes the time as the weight, this is the weight of a certain condition, such as the number of times by the top, then we need our sorted set to go, set the value you want to sort into the score of sorted set, Set the specific data to the corresponding value, each time only need to execute a zadd command.

3. Applications that require precise setting of expiration time
For example, you can set the score value of the sorted set to the timestamp of the expiration time, then you can simply sort through the expiration time, and periodically purge out-of-date data, not only to clear the expired data in Redis, You can think of this expiration time in Redis as an index to the data in the database, use Redis to find out what data needs to be deleted, and then delete the corresponding records from the database exactly.

4. Counter Application
Redis commands are atomic, and you can easily use the INCR,DECR command to build a counter system.

5.Uniq operation to get all data rows for a certain period of time
This is the most appropriate set data structure to use Redis, just to constantly throw it into set, set means set, so it automatically takes weight.

6. Real-time system, anti-spam system
With the Set function mentioned above, you can know whether an end user has performed an operation, can find a combination of its operations to analyze statistical comparisons, and so on. There is nothing to be done, only unexpected.

7.pub/sub Building a real-time messaging system
Redis's pub/sub system can build real-time messaging systems, such as many examples of real-time chat systems built with Pub/sub.

8. Build a queue system
Using list, you can build a queue system, and you can even build a prioritized queue system using sorted set.

9. Caching
This needless to say, performance is better than memcached, data structure is more diversified.

Http://blog.nosqlfan.com/html/2235.html

Extended reading:
Redis replication and scalable cluster setup, Http://www.infoq.com/cn/articles/tq-redis-copy-build-scalable-cluster
Redis Application scenario, http://blog.csdn.net/hguisu/article/details/8836819

Http://www.baidu.com/s?wd=redis Usage Scenarios
Http://www.sogou.com/web?query=redis Usage Scenarios
Https://www.so.com/s?q=redis Usage Scenarios
Http://blog.nosqlfan.com/topics/redis

Redis Usage Scenarios