A dht crawler in python
I recently write a dht crawler in python.
What a dht? well, you can think of it as a protocol as we treated the bt.
And I am not gonna tell you that I can’t understand it before I read the source code in python that implements it.
LOL…It is the truth, the paper sometimes is just hard to read.
I call it the “bt-share”, meaning that it provides the seeds search service for people.
It composite three parts: the web interface, the dht protocol crawler, and the bt downloader.
The basic structure is shown below:
It is inspired by the dht-crawler written in erlang. I draw a pic of its structure.