A few suggestions to make Redis play a bigger role in your system

"51CTO headline" Redis differs in many ways from other database solutions: It uses memory to provide primary storage support, and only uses hard disk for persistent storage; its data model is unique and is single-threaded. Another big difference is that you can use the capabilities of Redis in your development environment, but you don't need to go to Redis.

Turning to Redis is certainly desirable, and many developers have made Redis the preferred database from the outset, but it is not easy to replace the database framework if your development environment is already set up and the application is already running on it. In addition, in some applications that require large-capacity datasets, Redis is not suitable because its datasets do not exceed the memory available to the system. So if you have big data applications and mostly read access patterns, Redis is not the right choice.

However, one of the things I like about Redis is that you can incorporate it into your system, which can solve a lot of problems, such as tasks that your existing database is slow to handle. This allows you to optimize with Redis or create new features for your app. In this article, I want to explore how to add Redis to an existing environment and use its primitive commands to solve some of the common problems encountered in traditional environments. In these cases, Redis is not a preferred database.

Show the latest list of items

The following statement is often used to show the latest items, and with more data, the query will undoubtedly become slower.

SELECT * from foo WHERE ... ORDER by Time DESC LIMIT 10

In a web app, queries such as "list up-to-date replies" are common, which often leads to extensibility issues. This is frustrating, because the project was created in this order, but it had to be sorted in order to output it.

A similar problem can be solved with redis. For example, one of our web apps wants to list the latest 20 reviews posted by users. We have a "show all" link on the side of the latest comment, and you can get more comments when you click on it.

We assume that each comment in the database has a unique incrementing ID field.

We can use pagination to make page and comment pages, using the Redis template:

-Each time a new comment is published, we add its ID to a redis list:

Lpush latest.comments <ID>

-We crop the list to a specified length, so Redis only needs to save the latest 5,000 comments:

LTRIM latest.comments 0 5000

-Each time we need to get the range of items for the latest review, we call a function to complete (using pseudo code):

FUNCTION get_latest_comments (start,num_items): Id_list = Redis.lrange ("latest.comments",start, start+num_items-1) IF id_list.length < Num_items id_list = sql_db (" SELECT ...    ORDER by Time LIMIT ...") End RETURN Id_list End

What we do here is very simple. Our latest ID in Redis uses a resident cache, which is always updated. But we've made a limit of no more than 5,000 IDs, so our Get ID function will always ask for Redis. Access to the database is required only if the Start/count parameter is out of range.

Our system does not "refresh" the cache in the traditional way, and the information in the Redis instance is always consistent. The SQL database (or other type of database on the hard disk) is only triggered when the user needs to get "very far" data, and the home page or the first comment page will not bother the database on the hard disk.

Delete and filter

We can use Lrem to delete comments. If the deletion is very small, the other option is to skip the entry of the comment directly and report that the comment no longer exists.

There are times when you want to attach different filters to different lists. If the number of filters is limited, you can simply use a different Redis list for each of the different filters. After all, there are only 5,000 items per list, but Redis is able to use very little memory to handle millions of items.

Leaderboard Related

Another common requirement is that data from a variety of databases is not stored in memory, so the performance of the database is not as good as the ability to sort by points and update them in real-time, almost every second.

Typically, for example, the leaderboard for online games, such as a Facebook game, based on the score you usually want:

-List Top 100 high-score contestants

-List A user's current global rankings

These operations are a piece of cake for redis, and even if you have millions of users, there will be millions of new points per minute.

The pattern is this, each time we get a new score, we use this code:

Zadd Leaderboard <score> <username>

You may replace username with a userid, depending on how you designed it.

Getting the top 100 high-score users is simple: Zrevrange leaderboard 0 99.

The global ranking of users is similar, only need: Zrank leaderboard <username>.

Sort by user poll and time

A common variant pattern of the leaderboard is, like Reddit or hacker news, that the news is sorted according to a score similar to the following formula:

Score = Points/time^alpha

So the user's vote will be the corresponding to dig out the news, but the time will follow a certain index to bury the news. Here's our pattern, of course the algorithm is up to you.

The pattern is this, starting with looking at items that might be up-to-date, such as 1000 of the news on the first page are candidates, so let's just ignore the others, which is easy to implement.

-Each time a new news post is posted, we add the ID to the list and use Lpush + LTRIM to ensure that only the latest 1000 items are removed.

-There is a background task to get this list and continue to calculate the final score for each of the 1000 news articles. The results are populated by the Zadd command in the new Order, and the old news is cleared. The key idea here is that the sort work is done by the background task.

Overdue Item Processing

Another common sort of item is sorting by time. We use Unix time as a score.

The pattern is as follows:

-Each time a new item is added to our non-Redis database, we add it to the sorted collection. Then we use the time attribute, Current_time and time_to_live.

-Another background task using Zrange ... Scores queries the sorted collection and takes out the latest 10 items. If the Unix time is found to have expired, delete the entry in the database.

Count

Redis is a good counter, thanks to Incrby and other similar commands.

I believe that you have tried many times to add new counters to your database to get statistics or display new information, but eventually you have to discard them because of write sensitivity.

Okay, now using redis doesn't have to worry anymore. With atomic increment (atomic increment), you can safely add a variety of counts, reset with Getset, or let them expire.

For example, this action:

INCR user:<id> EXPIRE user:<id> 60

You can figure out the number of page views that have recently been paused for up to 60 seconds between pages, and when the count reaches like 20 o'clock, you can show some banner hints, or anything else you want to show.

Specific items within a specific time period

Another is difficult for other databases, but the easy part of Redis is to count how many specific users have visited a particular resource during a particular period of time. For example, I want to know some specific registered users or IP addresses, how many of them have visited an article.

Every time I get a new Page view I just need to do this:

Sadd page:day1:<page_id> <user_id>

Of course, you might want to replace day1 with Unix time, such as timing ()-(Times ()%3600*24) and so on.

Want to know the number of specific users? You only need to use SCard page:day1:<page_id>.

Need to test if a particular user has access to this page? Sismember page:day1:<page_id>.

Real-time analysis of what is happening, for data statistics and prevention of spam, etc.

We've only done a few examples, but if you look at Redis's command set and combine it, you'll get a lot of real-time analytics, effective and very labor-efficient. Using the Redis Primitives command makes it easier to implement spam filtering systems or other real-time tracking systems.

Pub/sub

Redis's pub/sub is very, very simple, stable and fast to run. Supports pattern matching and enables real-time subscription and cancellation of channels.

Queue

You should have noticed that Redis commands such as List push and list pop are easy to perform queue operations, but can do more than that: for example, Redis also has a variant command of list pop, which blocks the queue when the list is empty.

Cache

The cache portion of Redis is worth writing a new article, and I'm just saying it briefly. Redis can replace memcached, so that your cache can only store data to be able to update data, so you no longer need to regenerate the data every time.

Redis can solve your problem!

Now you can use Redis to make it easier for users to make your system less complex and make your site more responsive. You don't need to change the existing database structure and use Redis to bring new things to your framework to accomplish tasks that were previously thought impossible/difficult or expensive.

Original address: http://antirez.com/post/take-advantage-of-redis-adding-it-to-your-stack.html

A few suggestions to make Redis play a bigger role in your system