Experience summary 21--crawl Web data, exchange rates, Htmlagilitypack

Find a lot of information on the Internet, PHP is much more, and then found a CSV file. The processing is very troublesome, the foreign site speed is very slow.

Finally, to fetch data on the page, I grabbed from the Bank of China, you can go to other sites crawl.

1. Analog Request URL.

String url = "Http://srh.bankofchina.com/search/whpj/search.jsp?"

pjname=1316 ";

HttpWebRequest request = webrequest.create (URL) as HttpWebRequest;
Request. Method = "GET";
Request. ContentType = "text/html; Charset=utf-8 ";
Request. useragent = "mozilla/5.0 (Windows NT 6.1; WOW64; rv:30.0) gecko/20100101 firefox/30.0 ";
HttpWebResponse response = Request. GetResponse () as HttpWebResponse;
Stream Myresponsestream = Response. GetResponseStream ();
StreamReader Mystreamreader = new StreamReader (Myresponsestream, Encoding.UTF8);
string result = Mystreamreader.readtoend ();

2, the HTML processing.

Use Htmlagilitypack for assistance.

HTMLDocument doc = new HTMLDocument ();
Doc. loadhtml (result);
foreach (Htmlnode node in Doc. Documentnode.selectnodes ("//td"))
{
if (node. InnerText = = "USD")
{
result = node. NextSibling.NextSibling.InnerText;
Break
}
}

Specific data location. You can use F12 to find it first. Then use the tool to find, specific documents to check, this method temporarily get to USD-CNY exchange rate.

Experience summary 21--crawl Web data, exchange rates, Htmlagilitypack