The design of key-value pairs in Redis

The rich data structure makes the design of Redis very interesting. Unlike relational databases, Dev and DBAs require deep communication, review each line of SQL statements, and unlike memcached, do not require DBA involvement. Redis DBAs need to be familiar with data structures and understand usage scenarios.

Here are some common examples of the KV database to talk about the design of key values, and with the relational database to do a comparison, to discover the deficiencies of the relationship type.

User Login System

A system that records user login information, we simplify the business and leave only one table.

The design of relational database

Mysql> SELECT * from login; +---------+----------------+-------------+---------------------+ | user_id | name | Login_times | Last_login_time | +---------+----------------+-------------+---------------------+ | 1 |           Ken Thompson | 5 | 2011-01-01 00:00:00 | | 2 |           Dennis Ritchie | 1 | 2011-02-01 00:00:00 | | 3 |           Joe Armstrong | 2 | 2011-03-01 00:00:00 | +---------+----------------+-------------+---------------------+

USER_ID the primary key of the table, name indicates the user name, Login_times indicates the number of times the user logged in, Login_times will increase automatically each time the user logs on, and Last_login_time is updated to the current time.

Design of Redis

Relational data into the KV database, my method is as follows:

Key table name: Primary KEY value: Column name

Value column values

Generally use a colon as a separator, which is an unwritten rule. For example, in the Php-admin for Redis system, is the default colon segmentation, so user:1 user:2 key will be divided into a group. As a result, the above relational data are converted into KV data and recorded as follows:

Set Login:1:login_times 5 Set Login:2:login_times 1 Set Login:3:login_times 2 Set Login:1:last_login_time 2011-1-1 Set Login:2:last_login_time 2011-2-1 Set Login:3:last_login_time 2011-3-1 Set Login:1:name "Ken Thompson" Set Login:2:name "Dennis Ritchie" Set Login:3:name "Joe Armstrong"

This allows you to obtain or modify the user's logon count and last logon time and name by getting, set, if the primary key is known.

The general user is unable to know their own ID, only know their own user name, so there must be a mapping from name to ID, where the design is different from the above.

Set "Login:ken thompson:id" 1 Set "Login:dennis Ritchie:id" 2 Set "Login:joe Armstrong:id" 3

So each time the user logs on the business logic is as follows (Python version), R is the Redis object, and name is the user name that has been learned.

#获得用户的id UID = r.get ("login:%s:id"% name) #自增用户的登录次数 ret = R.INCR ("login:%s:login_times"% uid) #更新该用户的最后登录时间 ret = R.set ("login:%s:last_login_time"% uid, Datetime.datetime.now ())

If the requirement is only a known ID, update or get a user's last login time, the number of logins, relationship and KV database is no different. One through the Btree PK, one through the hash, the effect is very good.

Suppose you have the following requirements to find the most recently logged on n users. Developers to see, or relatively simple, a SQL fix.

　　

SELECT * FROM Login ORDER BY last_login_time desc limit N

After the DBA understands the requirements, build an index on the last_login_time, taking into account that if the table is larger later. The execution plan accesses N records from the far right of the index Leafblock, then returns the table n times, which works fine.

After two days, another demand, need to know the number of people who log in most. How does the same type of relationship work? Dev says simple

SELECT * FROM Login ORDER BY login_times desc limit N

The DBA looks and builds an index on the login_time. Do you think there is a problem, each of the fields on the table is a well-established guide.

The uncertainty of the data storage of relational database is the source of the problem, the data has only one storage method, that is the heap table arranged by rows. A unified data structure means that you have to use an index to change the access path of the SQL to quickly access a column, and the increase in access paths means that you have to use statistical information to assist, so a lot of problems arise.

No index, no statistical plan, no implementation plan, this is the KV database.

How to meet the above needs in Redis? In order to find the latest n data requirements, the list of backward features is very suitable. We add a piece of code after the above login code, maintain a login list, control his length, so that the inside forever save is the most recent n login users.

#把当前登录人添加到链表里 ret = R.lpush ("Login:last_login_times", UID) #保持链表只有N位 ret = Redis.ltrim ("Login:last_login_times", 0, N-1)

This requires the ID of the most recent logged-in person, such as the following code

Last_login_list = R.lrange ("Login:last_login_times", 0, N-1)

In addition, the number of people who have the most logins, for the sort, the table of such requirements, sorted set is very suitable, we put the user and the number of logins in a uniform storage in a sorted set.

Zadd Login:login_times 5 1 Zadd Login:login_times 1 2 Zadd Login:login_times 2 3

So if a user logs in and maintains an additional sorted set, the code

#对该用户的登录次数自增1 ret = R.zincrby ("Login:login_times", 1, UID)

Then how to get the most users of the number of logins, in reverse order to take the rank of nth users can

ret = R.zrevrange ("Login:login_times", 0, N-1)

As you can see, Dev needs to add 2 lines of code, and DBAs don't need to think about indexes or anything.

Tag system

tag in the Internet application is particularly common, if the traditional relational database to design a bit nondescript. Let's look for examples of books to see the advantages of Redis in this area.

The design of relational database

Two tables, a book of details, a tag table, indicating the tag of each one, there are multiple tag.

Mysql> select * from book; +------+-------------------------------+----------------+ | ID | name | Author | +------+-------------------------------+----------------+ | 1 | The Ruby Programming Language | Mark Pilgrim | | 1 | Ruby on Rail | David Flanagan | | 1 | Programming Erlang | Joe Armstrong | +------+-------------------------------+----------------+ Mysql> select * from tag; +---------+---------+ | tagname | book_id | +---------+---------+ |       Ruby | 1 | |       Ruby | 2 | |       web | 2 | |       Erlang | 3 | +---------+---------+

If there is such a need, the lookup is Ruby and the Web book, what if the relational database is handled?

Select B.name, b.author from tag t1, Tag T2, book B where t1.tagname = ' web ' and t2.tagname = ' ruby ' and t1.book_id = t2.book_id and b.id = t1.book_id

The tag table is associated 2 times with the book, and this SQL is still more complex, if the requirement is Ruby, but not web books?

Relational data is not really appropriate for these collection operations.

Design of Redis

First book data must be stored, and the same as above.

Set Book:1:name "The Ruby programming Language" Set book:2:name "Ruby on Rail" Set book:3:name "Programming Erlang" Set Book:1:author "Mark Pilgrim" Set book:2:author "David Flanagan" Set book:3:author "Joe Armstrong"

Tag table we use collections to store data because collections are good at intersection, and set

Sadd Tag:ruby 1 Sadd Tag:ruby 2 Sadd Tag:web 2 Sadd Tag:erlang 3

So, the book that belongs to Ruby and web?

Inter_list = Redis.sinter ("Tag.web", "Tag:ruby")

A book that belongs to Ruby but does not belong to the Web?

Inter_list = Redis.sdiff ("Tag.ruby", "Tag:web")

A collection of Ruby and web-owned books?

Inter_list = Redis.sunion ("Tag.ruby", "Tag:web")

It's not easy enough.

From the above 2 examples can be seen in some scenarios, relational database is not very appropriate, you may be able to design a system to meet the needs, but always feel strange, there is a kind of rote feeling.

In particular, login system This example, frequent indexing for the business. In a complex system, DDL (index creation) is likely to change the execution plan. causing other SQL to adopt different execution plans, complex old systems, the problem is difficult to predict, SQL strange. It's hard to ask the DBA to understand all the SQL in this system. This problem is particularly acute in Oracle, which every DBA has encountered. For MySQL-like systems, DDL is inconvenient (although there are now methods for online DDL). When the DBA got up in the morning and operated at the low peak of the business, I did little. This requirement is well handled in the Redis, and the DBA simply estimates the capacity.

The future OLTP system should be a close combination of KV and relational type.