Experience summary 21--crawl Web data, exchange rates, Htmlagilitypack

Find a lot of information on the Internet, PHP more, and then find a CSV file, processing is very troublesome, foreign sites slow.

Finally, to fetch data on the page, I grabbed from the Bank of China, you can go to other sites crawl.

1. Analog Request URL.

String url = "http://srh.bankofchina.com/search/whpj/search.jsp?pjname=1316";

HttpWebRequest request = webrequest.create (URL) as HttpWebRequest;
                request. Method = "GET";
                request. ContentType = "text/html; Charset=utf-8 ";
                request. useragent = "mozilla/5.0 (Windows NT 6.1; WOW64; rv:30.0) gecko/20100101 firefox/30.0 ";
                HttpWebResponse response = Request. GetResponse () as HttpWebResponse;
                Stream Myresponsestream = response. GetResponseStream ();
                StreamReader mystreamreader = new StreamReader (Myresponsestrea M, Encoding.UTF8);
                String result = Mystreamreader.readtoend ();

2, the HTML processing. Use Htmlagilitypack for assistance.

HTMLDocument doc = new HTMLDocument ();
Doc. loadhtml (result);
foreach (Htmlnode node in Doc. Documentnode.selectnodes ("//td"))
{
if (node. InnerText = = "USD")
{
result = node. NextSibling.NextSibling.InnerText;
Break
}
}

Specific data location, you can first use F12 to find, and then use the tool to find, detailed documents themselves to check, this method temporarily get to USD-CNY exchange rate.

Experience summary 21--crawl Web data, exchange rates, Htmlagilitypack