[Crawler learning notes] C # DNS resolution module (semi-finished product) based on ARSoft. Tools. Net ),

[Crawler learning notes] C # DNS resolution module (semi-finished product) based on ARSoft. Tools. Net ),

   
Recently, I am doing a crawler job. What I learned today is about the creation of the DNS resolution module. The library used is ARSoft. Tools. Net. It is a very powerful open-source DNS control library, including. Net SPF validation, SenderID validation, and DNS Client and DNS Server interfaces. You can use this interface to easily implement DNS client and server resolution. Project address: Workshop /.
 

   
First introduce the Nuget package:
    
 
Install-Package ARSoft. Tools. Net
 
    
 
 

   
The specific implementation is as follows:
         
/// <Summary> /// DNS resolution /// </summary> /// <param name = "dnsServer"> DNS Server IP address </param> /// <param name = "timeOut"> resolution timeOut </param> /// <param name = "url"> resolution url </param> /// <param name = "isSuccess "> whether the resolution is successful </param> // <returns> the resolved IP information </returns> public static IPAddress DnsResolver (string dnsServer, int timeOut, string url, out bool isSuccess) {// initialize DnsClient. The first parameter is the IP address of the DNS server, and the second parameter is the timeOut value var dnsClient = new DnsClient (IP Address. Parse (dnsServer), timeOut); // resolve the domain name. Send the domain name request to the DNS server for resolution. The first parameter is the domain name to be resolved, the second parameter is // resolution type, RecordType. A Is IPV4 type // DnsMessage dnsMessage = dnsClient. resolve ("www.sina.com", RecordType. a); var s = new Stopwatch (); s. start (); var dnsMessage = dnsClient. resolve (DomainName. parse (url); s. stop (); Console. writeLine (s. elapsed. milliseconds); // If the returned result is null or an error exists, the request fails. If (dnsMessage = null | (dnsMessage. ReturnCode! = ReturnCode. NoError & dnsMessage. ReturnCode! = ReturnCode. NxDomain) {isSuccess = false;} // cyclically traverses the returned results and adds the returned IPV4 Record to the result set List. If (dnsMessage! = Null) foreach (var dnsRecord in dnsMessage. answerRecords) {var aRecord = dnsRecord as ARecord; if (aRecord = null) continue; isSuccess = true; return aRecord. address;} isSuccess = false; return null ;}
  

  
Call method:
      
bool isSuccess;IPAddress ip = DnsResolver("223.5.5.5", 200, "shaoweicloud.cn", out isSuccess);if (isSuccess)    Console.WriteLine(ip);
    
 

 

  
We can further encapsulate it to obtain the DnsResolver class:
      
Using System; using System. collections. generic; using System. diagnostics; using System. net; using ARSoft. tools. net; using ARSoft. tools. net. dns; namespace Crawler. protocol {public class DnsResolver {public TimeSpan {get; set;} public string Url {get; set;} public List Record {get; set;} public string DnsServer {get; set;} public int TimeOut {get; set;} public ReturnCode {ge T; set;} public bool IsSuccess {get; private set;} public DnsResolver (string url, string dnsServer = "223.5.5.5", int timeOut = 200) {Url = url; dnsServer = dnsServer; TimeOut = timeOut; Record = new List (); Dig ();} public void Dig () {// initialize DnsClient. The first parameter is the IP address of the DNS server, the second parameter is the timeout value var dnsClient = new DnsClient (IPAddress. parse (DnsServer), TimeOut); var s = new Stopwatch (); s. start (); // resolve the domain name. Send the domain name request to the DNS server for resolution. The parameter is the domain name var dnsMessage = dnsClient to be resolved. resolve (DomainName. parse (Url); s. stop (); TimeSpan = s. elapsed; // If the returned result is null or an error exists, the request fails. If (dnsMessage = null | (dnsMessage. ReturnCode! = ReturnCode. NoError & dnsMessage. ReturnCode! = ReturnCode. NxDomain) IsSuccess = false; // cyclically traverses the returned results and adds the returned IPV4 Record to the result set List. If (dnsMessage! = Null) foreach (var dnsRecord in dnsMessage. answerRecords) {var aRecord = dnsRecord as ARecord; if (aRecord = null) continue; IsSuccess = true; Record. add (aRecord);} if (dnsMessage! = Null) ReturnCode = dnsMessage. ReturnCode ;}}}
  

  
Call method:
      
DnsResolver dns = new DnsResolver("shaoweicloud.cn");if (dns.IsSuccess)    Console.WriteLine(dns.Record[0]);
  

  
At this point, the DNS resolution module has basically ended. Why is the title labeled as a semi-finished product, this is because I want to implement an information cache mechanism based on the TTL that is parsed to the DNS information based on the Basic DNS resolution function to reduce unnecessary duplicate queries. What methods are currently being considered, subsequent implementations will be updated.