Load Balancing policies in Distributed Environments

In a distributed system, multiple servers are deployed for the same service. Each server is called a service node (instance ). Server Load balancer uses some load balancing policies to evenly distribute service requests to each node, so that the entire system can support massive requests. This article describes some simple Load Balancing policies.

Round-Robin

Simple round robin. Record a selected location and adjust it to the next node upon each request:

curId = ++curId % nodeCnt

Random selection

Randomly select from all nodes:

id = random(nodeCnt);

Local priority

The visitor accessing the background service may be an integrated service or a proxy. If the background service node happens to have a node deployed on the local machine, it can be used first. When the local node is not found, you can continue with the round-robin policy:

if (node->ip() == local_ip) {    return node;} else {    return roundRobin();}

Once it is traversed to the local node, the subsequent requests will continue to fall to the local node. Therefore, we can add some weight mechanisms to ensure that the node on the local machine is preferentially selected but not always selected. For example:

// initialcur_weight = 100;...// select nodecur_weight -= 5;if (cur_weight <= 0)    cur_weight = 100;if (cur_weight > 50 && node->ip() == local_ip) {    return node;} else {    return roundRobin();}

Data center priority

Service nodes may be deployed in multiple data centers. Sometimes cross-data center services need to be considered. SameLocal priorityThe policy is similar. In this IDC, priority is given to service nodes located in the same IDC. If the request is sent from the front-end service in the data center, the front-end must carry the ID of the data center in the request parameters.

In the data structure corresponding to the service node, it is best to organize the data according to the data center.

The priority policy of the IDC is actually the first process selected by the node. It can filter out non-IDC nodes and then input various node selection policies. The number of nodes can also be considered here. If the number of nodes in the data center is too small, this policy is not used to avoid serious traffic imbalance.

Weighted Round-Robin

Weighted Round Robin. Compared with normal round robin, each node in this policy has its own weight, and the priority is greater than that of the node. The weight can be pre-configured based on machine performance. Excerpt the online algorithm:

Suppose there is a group of servers s = {S0, S1 ,..., Sn-1}, w (SI) indicates the weight of the server Si, an indication variable I indicates the server selected last time, the indication variable CW indicates the weight of the current scheduling, max (s) indicates the maximum weight of all servers in the Set S, and gcd (s) indicates the maximum public weight of all servers in the Set S. Variable I is initialized to-1, and CW is initialized to zero. While (true) {I = (I + 1) mod n; if (I = 0) {CW = CW-gcd (s); If (cw <= 0) {CW = max (s); If (Cw = 0) return NULL ;}} if (w (SI) >=cw) return Si ;}

After traversing all nodes, the weight is reduced to 0 and then the node starts again. In this way, more nodes with higher weights can be selected.

Consistent hash

Consistent hash. Consistent hash is used for requests distributed on each node in a distributed environment and does not change because of the addition of nodes (resizing) or reduction of nodes (node downtime. If each service node has its own cache, it saves the node's response to the request. Under normal circumstances, these caches can be used very well, that is, the cache hit rate is high.

If a node is unavailable and our selection policy is based on the fair choice of all nodes, requests that have been allocated to node A may be distributed to Node B, this makes the cache on node A more difficult to hit. In this case, consistent hash can be used to solve the problem.

The basic idea is to find a node that is not less than the hash value corresponding to the request clockwise during the node selection interval. Many virtual nodes are added in this range. Each virtual node is equivalent to a reference of a physical node, which converts a physical node into a hash value range. The hash value range does not change because of adding or reducing nodes. For a request, it will always fall into this range and be allocated to the original node.

As for this unavailable node, the requests on it will be evenly distributed to other nodes.

Extract A piece of code from the Internet:

// When a physical node is added, many virtual node templates <class node, class data, class hash> size_t hashring <node, Data, hash> are added :: addnode (const node & node) {size_t hash; STD: String nodestr = stringify (node); For (unsigned int r = 0; r <replicas _; r ++) {hash = hash _ (nodestr + stringify (r )). c_str (); ring _ [hash] = node; // physical and virtual nodes are saved in a STD: Map} return hash ;} // select the node corresponding to the data. The data can be the request template <class node, class data, class hash> const node & hashring <node, Data, hash> :: getnode (const Data & Data) const {If (ring _. empty () {Throw emptyringexception ();} size_t hash = hash _ (stringify (data ). c_str (); // hash the request typename nodemap: const_iterator it; // look for the first node> = hash it = ring _. lower_bound (hash); // find the first node that is not less than the request hash if (IT = ring _. end () {// wrapped around; get the first node it = ring _. begin ();} return it-> second ;}

Reference consistent hash algorithm (Consistent hashing), consistent hash Ring

Address: http://codemacro.com/2014/08/25/lb-policy/
Written by Kevin Lynx posted athttp: // codemacro.com

Load Balancing policies in Distributed Environments