Sina Weibo Redis service platform from the application perspective (1)

You can briefly describe the actual situation of the apsaradb for Redis platform.

2200 + 500 billion commands/day 50 billion Read/day Write/day

18 TB + Memory

500 + Servers in 6 IDC 2000 + instances

It should be a big Redis platform at home and abroad. Today we will talk about the Redis service platform from the application perspective.

Redis application scenarios

1. Counting count)

The Application of counting is described in more detail in another article, the optimization of counting scene http://www.xdata.me /? P = 262.

It is foreseeable that many people think that the cost of storing all the counts in the memory is very high. Here I use a chart to express my point of view:

In many cases, you may imagine that the memory-only solution will be very costly, but the actual situation may be somewhat different:

1. for applications with certain throughput requirements, COST will apply for DB and Cache resources separately. Many students who are worried about DB write performance will also take the initiative to log DB updates to the asynchronous queue, the utilization of these three resources is generally not too high. After calculating the resources, you are surprised to find that the pure memory solution will be simpler!

2. KISS principle. This is very friendly for development. I only need to establish a connection pool, so I don't have to worry about Data Consistency Maintenance and do not need to maintain asynchronous queues.

3. The risk of Cache penetration. If the backend uses the database, it will certainly not provide high throughput capacity. If the cache is down and not properly handled, it will be a tragedy.

4. Most of the initial storage requirements are small.

2. Reverse cache)

In the face of the hot spots that often appear on Weibo, such as the recent popularity of Short chains, there are tens of thousands of people clicking and jumping in a short time, and there will often be some demands here, for example, we have displayed different content or information to determine the user level, whether there are some account bindings, Gender hobbies, and so on during a quick jump.

Generally, Memcache + Mysql is used. When the call id is valid, it can support large throughput. However, when the call id is uncontrollable and many spam users call it, memcache will penetrate into the Mysql server in large quantities because memcache does not hit the call id. This results in a high connection length and a low overall throughput, slow response time.

Here, we can use redis to record all user judgment information, such as string key: uid int: type, for reverse cache. After a user quickly obtains information such as his or her level in redis, go to the Mc + Mysql layer to obtain full information.

Of course, this is not the optimal scenario. For example, using Redis for bloomfilter may save more memory usage.

3. Top 10 list

Product Operations always show you the top list with the latest, most popular, highest click rate, and highest activity. If you use MC + MySQL to maintain a list of frequently updated items, the possibility of cache invalidation will be relatively high. In view of the small memory usage, using Redis for storage is also quite good.

4. Last Index

Recent user access records are also a good application scenario for redis list. lpush lpop automatically expires old login records, which is very friendly for development.

5. Relation List/Message Queue

Here we put the two functions at the end, because these two functions encountered some difficulties in the real problem, but at a certain stage they did solve many of our problems, so here we will only explain them.

Pinterest uses Redis to store social graph information:

Http://blog.gopivotal.com/case-studies-2/using-redis-at-pinterest-for-billions-of-relationships

Message Queue writes and consumes queues through the lpop and lpush interfaces of the list. Because of its good performance, it can also solve most problems.

6. Fast transaction with Lua

The expansion of Redis Lua has actually brought more application scenarios to Redis. You can write several command combinations as a small non-blocking transaction or update logic, such: when receiving the message push. add an unread Dialog for yourself. add an unread message to your private message. finally, send the sender a receipt to complete the PUSH message. This layer of logic can be fully implemented on the Redis Server.

However, it should be noted that Redis will record all content of lua script in aof and send it to slave, which will also be a great overhead for the disk and nic.

7. Instead of Memcache

Many tests and applications have proved that,

1. Redis does not lag behind Memcache in terms of performance, while the single-thread model has brought strong scalability to Redis.

2. In many scenarios, the memory overhead of apsaradb for Redis for the same data set is smaller than that allocated by slab of apsaradb for Memcache.

3. The data synchronization function provided by Redis is actually a powerful extension of the cache function.