Record the main functional steps of the DHT network, and the main functional steps of the dht

Record the main functional steps of the DHT network, and the main functional steps of the dht

The basic operations of the DHT network are basically implemented by searching relevant information without stopping. It is recorded for future reference.

Protocol

Kad defines the interaction protocol between nodes. These protocols support the implementation of distributed information storage in the entire DHT network. These protocols are transmitted over UDP. The Protocol format uses a bencode encoding method to encode protocol data. Bencode is a text format encoding that is also used for information encoding in the seed file.

For the specific format of the Kad Protocol, see the definition of BitTorrent: DHT Protocol. These protocols include ping, find_node, get_peer, and announce_peer. In some documents, the names of these requests are different. For example, announce_peer is also called store and get_peer is called find_value. All four types of requests have corresponding response messages. The most important message isget_peerTo find the peer list corresponding to a resource in the network.

It is worth mentioning that all these requests, including various responses, can be used to build route tables for nodes that process the message. Because a route table stores node information in the network in essence.

Ping

Used to determine whether a node is online. This request is mainly used to update the route table.

Find_node

Used to find a node to obtain its address information. When a node receives the request, if the target node is not in its route table, K nodes close to the target node are returned. This message can be used to build a route table when the node starts. Build a route table using find_node, and query yourself from the DHT network. Then, the node that receives the query will always return to the list of other nodes. The queryer performs recursive query until the query fails. So when cannot the query continue? I am not quite clear about this. Each query results in a node set that is closer to the target node. Theoretically, after several recursive queries, the node closer to the target node cannot be found, because the nearest node is itself, but it is not completely added to the network. This means that all nodes will return an empty node set at the end of the query?

In fact, using find_node to build a route table and add the route to the DHT network without any delay is not important to me. The construction of the route table does not need to be completed at startup. In the future interaction with other nodes, the route table itself will be built slowly. In the initial stage, we try to use find_node to interact with other nodes. The biggest benefit is to let other nodes in the network know themselves as soon as possible.

Get_peer

Obtain the peer list corresponding to the resource through infohash of the resource. After obtaining the peer list of resources, the queryer can download the resources through these peers. The node that receives the request will search for the infohash in its own route table. If the list is included, the corresponding peer list will be returned. If no node exists, several nodes closer to the infohash node are returned. If the queryer receives a node list, it recursively searches for the node list. This process is the same as that of find_node.

It is worth noting that the response message of get_peer carries a token, which will be used for later announce_peer requests.

Announce_peer

The main purpose of this request is to notify other nodes to download a resource by themselves. This message is used to build a peer list of resources in the network. When a P2P client that has joined the DHT network starts to download resources through the seed file, it first queries the peer list of the resource in the network. This process is completed through get_peer. When a node returns a peer from get_peer, The queryer starts to download the node and then tells the node that the peer is returned through announce_peer.

Announce_peer carries the token in the Response Message of get_peer. In this regard, I have a question: in P2P, the DHT network introduction document mentions:

(Announce_peer) at the same time, it will send its peer information to the previous reporter and K nearest nodes in its k bucket to store the peer-list information

Whether the nearest node of K is closest to oneself or the node closest to the resource infohash, there is a token verification process when processing announce_peer messages. However, the corresponding token is not created in the K nodes. I have collected data through DHT in transmission. It can be proved that announce_peer messages will not only be sent to get_peer responders.

1. Insert a route table.
1) if the node is already in the routing table, update the node and return.
2) If the bucket is not full, insert and return.
3) If an invalid node is found, replace it and return.
4) if a suspicious node is found, the new node is saved to the cache. If the suspicious node is not pinged, The ping_node operation is sent and a result is returned.
5) Now, the bucket is full of good nodes. If your ID is not in this bucket, return.
6) divide the bucket into two halves. Jump to step 1 ).

2. KAD remote processing calls.
This part is divided into three types,
1) ping/pong operation.
Tid of all packages uses pg \ 0 \ 0
2) find_node operation.
All packages use fn \ 0 \ 0 tid.
3) get_peers/annouce_peer operation.
Tid remains unchanged in a recursive query of the same HASH.
There are only 3) recursive query operations mentioned in the DHT specification of bittorrent, 1) and 2) used only to maintain the route table without saving the status.

3. Timer processing:
To check the validity of nodes in the routing table (according to the specifications, only valid nodes should be saved in the routing table), if you perform node operations in the route table when performing the krpc operation in the code, the start time of the operation is pinged_time. the start time of the operation is used to determine whether the operation times out.

When expire_stuff_time times out, the following operations are performed:
1. Check the invalid nodes in the routing table (determined based on pinged_time) and delete the nodes.
2. Check whether the node used to save annoounce_peer is more than 30 minutes (this is not intended to be discussed in depth, so it is not parsed ).
3. Check that the recursive query operation times out.

Rotate_secrets_time timer.
Used to change the token every about 15 minutes (see the DHT Specification ).

Confirm_nodes_time timer.
Find the bucket that has not been active for a long time, and then refresh it by executing a find_node krpc operation.

Search_time timer.
The search_time timer is responsible for resending all requests in the case that all get_peers operations are not answered. (Note: The maximum number of outstanding krpc requests for the get_peers operation is 3)

The ping/pong operation used to maintain the route table:
When you try to insert a node, the bucket is found to be full. If a suspicious node exists, the ping_node operation is triggered. If the node does not respond, it will become invalid and be replaced.
 

 

The following describes how to access the DHT network.

References

• DHT Protocol
• DHT network in P2P
• Transmission DHT source code

At present, I have initially set up a small website SOSOBT.com and officially increased my understanding of the magnetic website.

Continue to record DHT-related information later.