How to make a piece of artifact like Btbook.net

These days btbook.net this search piece artifact website fire, let me this person who does not work, also want to do a come out, otherwise time is bad, my drafts station: Fastbot.me (just released a few hours, experience wait a few days to do)

Now let's talk about how this kind of search artifact is done:

First, implement a DHT crawler

As for how to achieve, I will not say more, I have a few articles here said.

1, vernacular explains the principle of DHT and its history

2, how to "raise" a DHT crawler

3, how to get the DHT crawler to go farther

4, me and the DHT crawler those fucking things

Take a moment to see the GitHub address of my two DHT crawlers.

DHT Crawler, the equivalent of the engine in the car, the technology is the most difficult to achieve, but also the most important, although you can use other DHT crawler, but to integrate into their own search artifact, or need a certain technology, write their own words, a greater sense of achievement.

Second, seed download

Seed download is also very troublesome things, if it is their own implementation of the agreement for seed download, download the seeds of special time, no good network bandwidth, do not think.

Directly say how to go to the Thunderbolt server to download seed files, and how to torrage.com directional collection to first fill a face.

With Wireshark bag analysis of the Thunderbolt, found that it is such a process: Get a magnetic link, take out the 40-byte Infohash, in their own seed server for HTTP requests to see if they have been cached, if the cache, direct download. If there is no cache, go to the DHT network to download. That's why you see thunder sometimes download seeds fast and slow phenomenon, fast because of direct to their own seed server download, slow because of a certain number of to the DHT network to send Get_peers request.

OK, as long as we realize the DHT crawler, get Infohash, you can go to the Thunder Seed Library to download. So what is the construction of an HTTP request to the Thunderbolt seed bank? Grab package Analysis Diagram I don't have screenshots, direct text description under.

Original INFOHASH:AD2150E029B61CAEE9EDC5F4E9B02CEE489788CD

Capital INFOHASH:AD2150E029B61CAEE9EDC5F4E9B02CEE489788CD

Request Address: Http://bt.box.n0808.com/AD/CD/AD2150E029B61CAEE9EDC5F4E9B02CEE489788CD.torrent

Header header information: referer:http://bt.box.n0808.com (must write, otherwise return error information)

As simple as that, be sure to note that ad is the Infohash first two characters, and the CD is the last two characters of Infohash.

If the Thunder seed server does not have the seed of your given infohash, that's even true, unless you're going to implement the seed download protocol.

Add: Listen to some of my "followers" say, do not know how to download the seeds through the infohash, here on some of the download through the thunder of the seeds, through the torrage.com download code example:

Basic functions for parsing seed files (torrent.py) #encoding: Utf-8 from time import time Def torrentinfo (torrentcontent): metadata = Torre ntcontent["Info"] info = {"Name": GetName (metadata), "Length": calclength (metadata), "Timesta MP ": Getcreatedate (torrentcontent)," Files ": Extrafiles (metadata)} return info def calclength (Metada  TA): length = 0 try:length = metadata["Length"] except KeyError:try:for file in
    
metadata["Files": Length = = file["Length"] except Keyerror:pass return length def extrafiles (metadata): Files = [] try:for file in metadata["files"]: Path = file["path
                    "] If Len (path) > 1:main = path[0] for f in path[1:]:
            Files.append ("%s/%s"% (main, F)) Else:files.append (path[0]) if files: return ' \ r \ n '. Join (Files) Else:return getName (metadata) except Keyerror:return getName (metadata) def Getnam
        E (metadata): Try:name = metadata["Name" if Name.strip () = "": Raise Keyerror except: Name = Getmaxfile (metadata) return name Def getmaxfile (metadata): Try:maxfile = Metadata ["Files"]  [0] for file in metadata["files": If file["Length"] > maxfile["Length"]: Maxfile = File name = maxfile["path"][0] return name except Keyerror:return "" Def getcreatedate ( torrentcontent): Try:timestamp = torrentcontent["Creation date"] except keyerror:timestamp = int (Time ()) return timestamp