codis--Solutions for Distributed Redis Services

codis--Solutions for Distributed Redis Services

The previously introduced Twemproxy is a Redis proxy, but it does not support dynamic scaling of clusters, while CODIS supports the dynamic increase or decrease of REDIS nodes, and the official Redis 3.0 begins to support cluster.

The biggest differences between Codis and Twemproxy are two:

• CODIS supports dynamic horizontal scaling, which can be done to increase or decrease redis instances without affecting the client's full transparency.
• Codis is written in the Go language and supports multi-threading, Twemproxy with C and only single thread. The latter also means that the performance of Codis on multicore machines will be better than Twemproxy;codis's worst response time may be greater due to the STW of the GC, However, go1.5 release will significantly reduce the time of STW; If you use only one CPU, the go language performance is not as good as C, so in some short connections rather than long connected scenarios, the entire system bottleneck may become the speed of the accept new TCP connection, when the performance of Codis may be worse than twemproxy.

The difference between Codis and Redis cluster:

The Redis cluster is based on smart client and no-center design, and the client must send the request directly to the corresponding node by the hash of the key. This means: The use of official cluster must wait for the corresponding language of the Redis driver to cluster support development and maturity; Client cannot use pipeline directly as a single machine to improve efficiency. To speed up multiple requests at the same time, you must implement asynchronous logic on the client side itself. Codis because of its central node, proxy-based design, for the client can be like a single-machine Redis to operate proxy (except some commands do not support), You can also continue to use pipeline and if the background Redis has more than one, the speed will be significantly faster than the pipeline of single Redis. At the same time Codis uses zookeeper as an aid, which means that only the Redis cluster requires additional machines to take ZK, but this is not a problem for many companies that have used ZK on other services:)

Codis is a distributed Redis solution developed by the Pea Pod Company, developed in the go language. For upper-level applications, there is no obvious difference between connecting to Codis Proxy and connecting to the native Redis Server (unsupported list of commands), Codis processing requests for forwarding, non-stop data migration, and so on. All the things behind, for the front of the client is transparent, you can simply think behind the connection is an infinite memory of the Redis service.

The Codis consists of four parts:

• Codis Proxy (Codis-proxy), handles client requests, supports REDIS protocol, so client access to Codis Proxy is no different from accessing native Redis;
• Codis Dashboard (codis-config), Codis management tool, supports adding/removing REDIS nodes, adding/removing Proxy nodes, initiating data migration, and so on. Codis-config itself with an HTTP server, will start a dashboard, users can directly in the browser to observe the Codis cluster running state;
• Codis Redis (Codis-server), a redis branch of Codis Project maintenance, based on 2.8.21 development, added slot support and atomic data migration directives;
• Zookeeper/etcd,codis relies on ZooKeeper to store the metadata of the data routing table and Codis-proxy node, and Codis-config commands are synchronized to each surviving ZooKeeper through Codis-proxy;

Codis supports differentiating between different products according to Namespace, with different product name products and no conflicting configurations.

Codis uses pre-sharding technology to achieve shard of data, which is divided into 1024 slots (0-1023) by default, and for each key, the Slot Id to which it belongs is determined by the following formula:

SlotID = CRC32 (key)% 1024

Each slot will have one and must have a specific server group ID to indicate which server group the data for this slot is provided by. The migration of data is also in slots.

Installation and Deployment

2. Install go;
4. Installing Codis

   Go get-u-D github.com/codislabs/codiscd $GOPATH/src/github.com/codislabs/codis make

6. Install zookeeper;
8. Start Dashboard

   Bin/codis-config Dashboard

10. Initialize slots, create slot-related information on ZK

    Bin/codis-config Slot Init

12. Start Codis-redis, just like the official Redis server method;
14. Adding Redis server group, each group exists as a Redis server group, only one master is allowed, and multiple Slave,group IDs support only integers greater than or equal to 1. For example: Add two server group, each group has two Redis instances, group IDs are 1 and 2, and the Redis instance is a master one from.

16.  bin/codis-config Server add 1  localhost: 6379   Masterbin /codis-config server add 1  Localhost:6380   Slavebin /codis-config Server add 2  localhost: Masterbin /codis-config server add 2  localhost:6480  slave 

     

17. The
18. sets the slot range for the server group service, such as a slot set to [0, 511] that is serviced by server Group 1, and the slot for number [512, 1023] is serviced by server Group 2

    bin/codis-config slot Range-set 0  511  Span style= "color: #800080;" >1   Onlinebin /codis-config slot Range-set 512  1023  2  online 

     

20. Start Codis-proxy,

    Bin/codis-proxy-c config.ini-l./log/proxy.log  --cpu=8 --addr=0.0.  0.0:19000 --http-addr=0.0. 0.0:11000

    The newly-launched Codis-proxy default is in the offline state, and then set the proxy to the online status, only the proxy in the online state will be available to the external service

    Bin/codis-config-c config.ini Proxy Online <proxy_name>  <----proxy ID, such as proxy_1

Data Migration (Migrate)

Secure and transparent data migration is an important feature provided by Codis and a place where Codis differs from static, distributed Redis solutions such as Twemproxy.

The smallest unit of data migration is key, and we have added some instructions in Codis Redis to implement key-based migrations, such as SLOTSMGRT (command list), and each time a random key from a specific slot is sent to another Codis Redis instance, the command will Recognize that the other party has received, while deleting the local k-v key value, return the number of the remaining key of this slot, the entire operation is atomic.

In the Codis-config management tool, the minimum unit of each migration task is the slot. For example, migrating the data for slots with slot ID [0-511] to server group 2, the--delay parameter indicates the number of milliseconds to sleep after each key migration, and the default is 0 for speed limit.

0 511 2 --delay=

The process of migration is secure and transparent to the upper business, the data is not lost, and the service is not aborted at the top.

Note that the interruption during the migration is possible, but if a task that is migrating a slot is interrupted, the next time you need to migrate the slot that is in the migration state, you cannot continue (that is, the migration program checks that only one slot is in the migration state at the same time).

Automatic rebalancing (auto rebalance)

Codis supports dynamically migrating slots based on instance memory to equalize data distribution

Bin/codis-config Slot Rebalance

Requirements:

• All codis-server must have the maxmemory parameter set;
• All slots should be in the online state, i.e. no migration tasks are being performed;
• All server group must have Master;

High Availability (HA)

Because the Codis proxy is stateless, it is easier to take multiple proxies to achieve high availability and scale-out.

For Java users, we can use the Jedis,jodis we have modified to implement the proxy layer of HA. It provides real-time access to the currently available proxy list by monitoring the registration information on ZK to ensure high availability, or load balancing by taking turns requesting all proxies. If you need an asynchronous request, you can use the Nedis that we developed based on Netty.

For Redis instances below, when a group's master hangs up, it should be clear to the administrator and manually, because this involves data consistency and other issues (Redis's master-slave synchronization is eventual consistency). Therefore, Codis does not automatically upgrade a slave to master. But we also provide a solution: Codis-ha. This is a tool that automatically switches master and slave through the Codis open API. The tool will go offline when it detects that Master is dead and choose one of the slave to continue to service master.

Note that when Codis upgrades one of the slave to master, other slave instances within that group do not automatically change state, and these slave will still attempt to synchronize data from the Old master. This results in inconsistent data between the new master and other slave in the group. Because Redis's slave of command switches to master, it discards all data on the slave, and full synchronization from the new master consumes the master resource. It is therefore advisable to manually operate it in an informed situation. Use the codis-config server add <group_id> <redis_addr> slave commands to refresh the state of these nodes. Codis-ha does not automatically refresh the status of other slave.

codis--Solutions for Distributed Redis Services