Implementation Based on redis distributed cache (Sina Weibo case study)

First, what is redis?

Redis is a memory-based, persistent log-type, key-value database high-performance storage system that provides APIs in multiple languages.

Second, Background

• Data structure requires more and more data structures, but memcache does not. This affects development efficiency.
• Performance requirements:
  Database read/write splitting (m/s)-> use multiple Server Load balancer instances in the database-> Add cache (memcache)-> switch to redis
• Solve the write problem:
  Horizontal Split: Split a table. Some users are placed in this table, and some users are placed in another table;

• Reliability Requirements
  The "Avalanche" problem of cache is confusing
  Cache is facing the challenge of rapid recovery

• Development Cost requirements
  The Consistency Maintenance Cost of cache and DB is getting higher and higher (clean up the database first and then clear the cache. No, it's too slow !)
  Development needs to keep up with the influx of product requirements
  The most expensive hardware cost is the database-level machines, which are several times more expensive than the front-end machines, mainly io-intensive and hardware-consuming;

• Complex maintainability
  Consistency Maintenance costs are getting higher and higher;
  When berkeleydb uses the B-tree, it will keep writing new data and there will be no re-organization of internal files; this will lead to an increasing number of files; when it is large, it needs to be archived, and archiving operations should be done on a regular basis;
  In this way, a certain down time is required;

Based on the above considerations, we chose redis

Third: Application of redis in Sina Weibo

Redis Introduction

1. Five data structures supported

Supports strings, hashes, lists, sets, and sorted sets.
String is a good storage method for counting storage. Sets is used to create an index database;

2. K-V storage vs K-V Cache

Sina Weibo currently uses 98% of persistent applications, 2% of which are caches, and more than 600 servers are used.
Persistent applications and non-persistent methods in redis are not very different:
The non-persistent TPS is 8-9 million, so the persistence is about 7-8 million TPS;
When using persistence, you must consider the ratio of persistence to write performance, that is, the ratio of memory size used by redis to hard disk write rate;

3. Active Community

Redis currently has more than 30 thousand lines of code, and the code writing is simplified. There are many clever implementations. The author has a technical cleanliness.
Redis has a high degree of Community activity, which is an important indicator to measure the quality of open-source software. In the early stage of open-source software, there is generally no commercial technical service support. If there is no active community for support, once a problem occurs, there is nowhere to ask for help;

Basic Principles of redis

Redis persistence (AOF) append online file:
Write log (AOF) to a certain extent and then merge with the memory. append and append, sequential write disk, has a very small impact on performance

1. Single-process instances

Redis uses a single process, so only one CPU is used for one instance during configuration;
When configuring, if you need to maximize the CPU usage, you can configure the number of redis instances to correspond to the number of CPUs, and the number of redis instances to correspond to the number of ports (8-core CPU, 8 instances, 8 ports ), to improve concurrency:
During the standalone test, the size of a single piece of data is 200 bytes, and the test result is 8 ~ 90 thousand transactions (TPS;

2. Replication

Process: data is written to the master and stored by the master to the slave RDB. The slave loads the RDB to the memory.
Save point: when the network is interrupted and connected, the data is continuously transmitted.
The first synchronization under Master-slave is full transmission, followed by incremental synchronization ;,

3. Data Consistency

There is a possibility of inconsistency between multiple nodes after long-term operation;
Develop two tool programs:
1. Periodic full check is performed for data with a large amount of data;
2. Check incremental data in real time for consistency;

The inconsistency caused by the failure to synchronize slave databases from the master database in a timely manner is called a latency problem;
For scenarios where consistency requirements are not so strict, we only need to ensure eventual consistency;
For the latency problem, we need to analyze the characteristics of the business scenario and add policies at the application level to solve the problem;
For example:
1. Newly registered users must first query the master database;
2. After the registration is successful, you need to wait for 3 seconds and then jump to the background to synchronize data.

 

Fourth: distributed cache Architecture Design

1. Architecture Design

Because redis is a single point, it must be used in projects and distributed by itself. The basic architecture diagram is as follows:

 

 

2. distributed implementation

Implements the distribution of keys corresponding to redis nodes through consistent hash of keys.

Implementation of consistent hash:

L hash value calculation: MD5 and murmurhash are supported. By default, murmurhash is used for efficient hash calculation.

L consistent implementation: Simulate the ring structure through the Java treemap to achieve even distribution

3. Client Selection

Jedis is mainly used to modify the partition module so that it supports Partitioning Based on bufferkey and different redis node information. Different shardinfo can be initialized, at the same time, the underlying implementation of the jedispool is modified to connect to the pool to support the construction of data keys and values. Different shardinfos are used to create different Jedis connection clients, achieve the partition effect for the application layer to call

4. Module description

L The dirty data processing module handles cache operations that fail to be executed.

L shield the monitoring module and monitor exceptions in Jedis operations. When an exception occurs at a node, you can control operations such as redis node removal.

The entire distributed module uses hornetq to remove abnormal redis nodes. You can also use the reload method to add new nodes. (This module can be easily implemented for new nodes)

The implementation of the above distributed architecture meets the needs of the project. In addition, some redis nodes can be set separately for cache data of some important purposes to set a specific priority. In addition, the design of the cache interface can also meet the requirements to implement basic interfaces and some special logical interfaces. Cas-related operations and some transaction operations can be implemented through the watch mechanism. (Refer to my previous redis transaction Introduction)

 

The above is an introduction based on the redis distributed architecture! However, the read and write operations in the application are all the same. Related writes are flushed or updated after application operations, with certain coupling. To enable read/write splitting and lower coupling between the cache module and the application, use MySQL BINLOG to refresh the cache. The following are precautions for refresh and Analysis Based on BINLOG.

 

 

 

Tsk:

Http://baike.baidu.com/view/4595959.htm? Fr = Aladdin

Http://blog.me115.com/2013/12/452

Http://blog.csdn.net/vhomes/article/details/8194670