Key-list Memory Data Engine introduction and use cases

Tags: redis, memlink, key-value, key-list, nosql

"Currently, Internet data is stored in two ways: relational databases or key values. However, these Internet businesses do not belong to these two data types. For example, the relationship between users in the social platform is a list, if you want to store data in a relational database, you need to convert it into a multi-row record format, which has a lot of redundant data, and each row needs to store some duplicate information. If the key value is used for storage, modification and deletion are troublesome. You need to read all the data before writing it "-Sina @ timyang

Key-value implements the list Function

If you use the value in key-value to store the list, you can only implement the simplest list function (sorted by ID or time, for example, using the memcache append or prepend protocol ). other list operations can only be performed on the client. The performance is poor. If the data size is large, the operation time is unacceptable, and concurrency also faces huge challenges ).

Currently, the mighty (internal R & D) Persistence Layer framework is used to operate the list. It is based on the memcache append prepend protocol to perform simple operations on the ID list, meeting the needs of most simple list scenarios. the disadvantage is that when there are many update operations that affect sorting, the cache hit rate will decrease sharply.

What is key-list?

The "value" corresponding to the key-list system key is a list (eg. set list) that can be used to operate a single item in the list. The ideal key-list requires the following features:

1. List can be massive and efficient

2. List can be ordered and can be dynamically adjusted.

Use Cases

List of topics and replies in the Forum

User follow list, user feed list, and user follow feed list in Weibo

Recent access list

Set operation: calculates the intersection Union difference set (sdiff sinter sunion)

Friend recommendation

Rankings

Open-source key-List System 

Redis

Redis is an open source, advancedKey-Value Store. It is often referred to asData structure serverSince keys can contain strings, hashes, lists, sets and sorted sets.

Redis is also included in the key-list system because redis supports list operations, as timyang says in his blog: different interpretations of the role of redis determine how you use redis.

Currently, Sina Alibaba Digg and other websites are using redis.

Memlink

MemlinkYestianyaCommunityA high-performance, persistent, and distributed key-list/queue Data Engine.

For more information, see:

Http://www.infoq.com/cn/news/2010/11/tianya-memlink

Http://code.google.com/p/memlink/

It is currently used in multiple products of Tianya.

 

In the future, I will continue to share with you the actual usage of redis in various application scenarios.

 

Related reading:

Memlink Performance Test and performance test comparison with redis and MySQL

 

Refer:

Http://www.infoq.com/cn/news/2010/11/tianya-memlink

Http://timyang.net/data/redis-misunderstanding/

Http://timyang.net/web/pagination/comment-page-1/

Http://redis.io

Http://code.google.com/p/memlink/