[Study of P2P algorithms in academic fields] chord algorithms in P2P

Author: gnuhpc
Source: http://www.cnblogs.com/gnuhpc/

A common problem of P2P is how to efficiently locate a node. That is to say, how can a node efficiently know which node in the network contains the data it is looking for? For example:

There are three typical solutions to this problem.

Napster: uses a central server to receive all queries, and the server tells where to download the required data. The problem is that a single point of failure on the central server causes the entire network to crash.

Gnutella: Uses message flooding to locate data. A message is sent to every node in the system until the required data is found. Of course, TTL is used to limit the number of forwarded messages in the network. The problem is that the number of messages is linearly related to the number of nodes, resulting in heavy network load.

SN: currently, most super nodes are used. The Sn stores the index information of nodes in the network. This is the same as that of the central server, but there are multiple SNS in the network, the index information will be transmitted in these SNS, so the chances of the entire system crash will be much lower. However, the Network may crash.

In the current research results, Chord, pastry, can, and tapestry are commonly used to construct a structured P2P Distributed Hash Table System (DHT ).

The main idea of DHT is: first, each file index is represented as a (K, v) pair. k is called a keyword and can be a file name (or other description of the file) is the IP address of the node where the file is stored (or other description of the node ). All the file index entries (that is, all (K, v) pairs) form a large file index hash table, as long as you enter the K value of the target file, you can find all the node addresses that store the file from this table. Then, the hash table of the above large files is divided into many local small blocks, and the local hash tables of these small blocks are distributed to all participating nodes in the system according to specific rules, each node is responsible for maintaining one of them. In this way, when a node queries a file, it only needs to route the query packet to the corresponding node (the hash table maintained by the node contains the (K, v) pair to be searched ).

The Chord algorithm introduced here is a P2P protocol that solves the problem of node locating in the network. It uses multiple nodes to jump to find the resources we are looking:

Let's first look at how it is implemented, and then summarize its features and operational features, as well as some implementations.

1. ChordBasic Elements

Node ID: NID (node identifier), which indicates a physical machine and a number in the m position (m must be large enough to ensure that the NID of different nodes is less likely to be ignored ), the IP address of the node machine is obtained by hashing.

Resource ID; kid (Key Identifiers), formerly the key ID, which actually represents a resource (because the key is hashed to a resource value ), therefore, in this article, we collectively refer to resource ID (this is more intuitive), a number in the M bit (m should be large enough to ensure that the probability of the same kid of different resources is small and negligible ), the key is obtained by hashing.

Common hash functions: Compared with common hash functions, adding and removing nodes have the least impact on the entire system. In addition, we will not repeat these advantages here. SHA-1 is used in chord for regular hash calculation.

Chord ring: chord ring. NID and kid are allocated to a 2 ^ m ring for resource allocation (to a node) and Node Distribution, and resource positioning (note: the ID on this ring is 0--2 ^ m-1 ). First, we will talk about resource allocation. The resource is allocated to the NID> = kid node. This node becomes the successor node of K, which is the first node in the clockwise direction from K on the ring, as successor (k ). Node Distribution places node N from large to small clockwise on this ring. For example, the following figure:

This is a ring with M = 6, with 10 nodes and 5 resources. The successor node of K10 is N14, that is, K10 is allocated to N14.

2. ChordResource positioning (Key location)

Resource locating is the core function of the Chord protocol. To facilitate understanding, we first introduce a simple resource locating method, and then introduce this scalable resource locating method.

Simple Method:

Consider the following scenario: node n searches for resources whose kid is ID. In this case, node n first queries whether the resource is located on the next node (find_successor ), it depends on whether the kid of resource K is between the NID of the node and the NID of the next node. If yes, the resource K is allocated to the next node, if not, initiate the same query on the next node and check whether the resource exists at the next node. In this iteration, use pseudo code to define this operation:

N. find_successor (ID)
If (ID cursor (N; successor])
Return successor;
Else
// Continue the query along the loop
Return successor. find_successor (ID );

For example:

Node n8 searches for the resource k54, and n8.find _ successor (k54) finds that the next node N14 does not match 54 operator (8; 14], so N14 initiates the same search, then, after a hop is made, the node n56 meets the condition of 54 Gb/s (51; 56], so it is known that the resource k54 is on the node n56.

In a ring with N nodes, this search method obviously requires n times in the worst case to obtain the location of the required resource. The number of searches is linearly related to the number of nodes. Obviously, this is not efficient, so chord uses Scalable Resource positioning to improve efficiency.

Scalability method:

A route table (finger table) with a maximum of m items (the number of digits where M is ID) is maintained on each node n to locate resources. The entry I of this table is the position of the subsequent node, at least after 2 ^ (I-1. Continue the above example:

In the route table of node n8, the column on the left contains the positions from n8 + 1 to n8 + 32 (2 ^ 5-1, the actual existing node corresponding to each position in the column on the right. For example, n8 + 1-n14 indicates that N14 is responsible for the resources in the first position after n8. This record has the following advantages:

Each node only contains information about a small part of the entire network.

Each node knows more about the locations of neighboring nodes. For example, the n8 node knows three locations for the N14 node, while the n21 node knows only one location.

The route table usually does not contain information that can be directly found on the successor node. It usually needs to be queried by other nodes.

When you search for a resource on a node, you first determine whether its successor node holds the resource. If not, you can directly search for the resource from the farthest distance from the route table of the node, check which node is closest to the node that holds the resource (jump after discovery). If it does not exist, it indicates that the node itself has the resource to be searched. This iteration continues.

For example, node n8 searches for the resource k54.

First, find the successor node N14 on n8, and find that k54 does not meet the requirements of 54 Gb/s (8; 14, then, you can directly search for the table items that meet this requirement on the n8 route table (from far to near). At this time, the n8 route table is:

We found that the longest n8 + 32--n42 in the route table is 42 Gb/s (8; 54], it indicates that the point n42 is closest to the node that holds the resource k54 (because n42 is the farthest from the node n8 in the route table), it will jump to the node n42 to continue searching. The successor node of n42 is n48, which does not meet the requirement of 54 Gb/s (42; 48]. This indicates that n48 does not hold resource 54. At this time, you can start searching on the route table of n42:

The route table of the n42 node is:

We start from far and near, and find that n42 + 8--n51 meets 51 bytes (42; 54], it means that n51 is closest to the node holding the resource k54, at this time, the node n51 will continue searching. The successor node of the n51 node is n56, which complies with the 54 slave node (51; 56]. At this time, the positioning is complete, and n56 holds the resource node k54.

Use pseudocode:

// Query node n.
N. find_successor (ID)
If (ID cursor (N; successor])
Return successor;
Else N0 = closest_preceding_node (ID );
Return n0.find successor (ID );
// Search the local table for the highest
// Predecessor of ID
N. closest_preceding_node (ID)
For I = m downto 1
If (finger [I] Then (N; id ))
Return finger [I];
Return N;

It has been proved that a resource can be found after O (log n) queries at most.

3. ChordAdd Node

Chord periodically performs stabilization on the backend of each node to check whether the subnode of the successor node is updating the items in the successor node and the route table.

There are three operations:
Join (N0): n joins a chord ring. It is known that there is a node n0.0.

Stabilize (): n queries the forward node P of its successor node to determine whether P should be a subsequent node of N. That is to say, when P is not N, it indicates that P is newly added, set the successor node of N to P.

Notify (N0): N0 notifies n of its existence. If n does not have a forward node or, N0 is closer to N than N's existing forward node, then n sets it as a forward node.

Fix_fingers (): modifies the route table.

For example:

This is the original structure:

Now n26 nodes need to be added to the system. First, it points to its n32, and then notifies n32. After receiving the notification, n32 marks n26 as its predecessor ). For example:

Then n26 modifies the route table, such:

The next time n21 runs stabilize (), it asks whether the n32 node of the successor node is itself or not. At this time, it is found that the n32 node of the frontend node is already n26:

Then n21 changed the successor node to n26, and notified n26 that it had been set as the successor node. After receiving the notification, n26 set n21 as its predecessor node.

This operation has two impacts:

1) Correctness: When a node is added to the system and a search occurs before the end of stabilization, the system has three states:

A. When all subsequent pointers and route table items are correct: the correctness is not affected.

B. The successor pointer is correct but the table item is incorrect: the search result is correct, but the speed is slightly slow (when many nodes are added to the successor of the target node and the target node ). For example:

C. The subsequent pointer and route table items are incorrect: When the query fails, the software at the upper level of chord will find that the data query fails and retry after a period of time.

To sum up, adding a node does not affect data search.

2) Efficiency: When stabilization is completed, the query efficiency will not be affected more than O (log n) time. When stabilization is not completed, performance is affected only when many nodes are added to the target node and the target node. It can be proved that the performance will not be affected as long as the route table is adjusted faster than the number of network nodes doubled.

4. ChordHandling of node failures

We can see that chord relies on the correctness of the successor pointer to ensure the correctness of the entire network. However, if N14, n21, and n32 both expire, n8 will not know that n38 is its new successor node. To prevent this, each node contains a list of subsequent nodes whose size is R. If a subsequent node fails, try other subsequent nodes in the list. It can be proved that in a network with a failure probability of 1/2, the search time is O (log n ).

5. ChordFeatures and Applications

Features: decentralization, high availability, high scalability, load balancing, and flexible naming.

Applications: Global File Systems, naming services, database Request Processing, Internet-level data structures, communication services, event notifications, and file sharing.

References:

Chord project URL: http://pdos.csail.mit.edu/chord/

Http://net.chinaunix.net/8/2008/07/28/1231438.shtml