C # uses XPath to parse Web pages

1, need to install library Htmlagilitypack, official website http://htmlagilitypack.codeplex.com/

//From FilevarDoc =Newhtmldocument ();d OC. Load (FilePath);//From StringvarDoc =Newhtmldocument ();d OC. loadhtml (HTML);//From WebvarURL ="http://html-agility-pack.net/";varweb =NewHtmlweb ();varDoc =web. Load (URL);//XPathvarnodes = Doc. Documentnode.selectnodes ("//*[@id =\ "body\"]");

XPath syntax: http://www.w3school.com.cn/xpath/xpath_syntax.asp

Other ways to parse a webpage: (Citation to: 78784464)

Method One: Use WebClient

Static voidMain (string[] args) {     Try{WebClient mywebclient=NewWebClient (); Mywebclient.credentials= CredentialCache.DefaultCredentials;//Gets or sets the network credentials that are used to authenticate requests to Internet resourcesbyte[] Pagedata= Mywebclient.downloaddata ("http://www.163.com "); //download data from a specified Web site         stringpagehtml = Encoding.Default.GetString (pagedata);//If you are using GB2312 to get a website page, use this sentence//string pagehtml = Encoding.UTF8.GetString (pagedata);//If you are using UTF-8 to get a website page, use this sentenceConsole.WriteLine (pagehtml);//Enter what you get in the console         using(StreamWriter SW =NewStreamWriter ("c:\\test\\ouput.html"))//writes the acquired content to the text{SW.         Write (pagehtml);                  } console.readline (); }     Catch(WebException webEx) {Console.WriteLine (webEx.Message.ToString ()); } }

Method Two: Use WebBrowser

WebBrowser Web =NewWebBrowser (); web. Navigate ("http://www.xjflcp.com/ssc/"); Web. DocumentCompleted+=NewWebbrowserdocumentcompletedeventhandler (web_documentcompleted);voidWeb_documentcompleted (Objectsender, WebBrowserDocumentCompletedEventArgs e) {WebBrowser Web=(WebBrowser) sender; HtmlElementCollection ElementCollection= Web. document.getElementsByTagName ("Table"); foreach(HtmlElement iteminchElementCollection) {File.appendalltext ("Kaijiang_xj.txt", item.             InnerText); }         }

Method Three: Use Httpwebrequest/httpwebresponse

HttpWebRequest Httpreq;  HttpWebResponse Httpresp; stringStrbuff =""; Char[] Cbuffer =New Char[ the]; intByteread =0; stringfilename =@"C:\log.txt"; ///defining write Stream Operations Public voidWritestream () {Uri Httpurl=NewUri (txturl.text);///The HttpWebRequest class inherits from the WebRequest and does not have its own constructor, which needs to be established by WebRequest's Creat method and forced type conversionHttpreq =(HttpWebRequest) webrequest.create (httpurl);///establish HttpWebResponse by HttpWebRequest's GetResponse () method, forcing type conversionsHttpresp=(HttpWebResponse) httpreq.getresponse ();///The GetResponseStream () method gets the HTTP response's data stream and attempts to get the page content specified in the URL///if the content of the Web page is successfully obtained, it is returned in System.IO.Stream form, and if the failure results in a protoclviolationexception error. In this correct practice, the following code should be put into a try block for processing. Easy to handle here .Stream Respstream =Httpresp.getresponsestream ();///The returned content is in stream form, so you can use the StreamReader class to get the contents of GetResponseStream andthe Read method of the StreamReader class reads the contents of each line of the source code of the Web page sequentially until the end of the line (the encoded format read: UTF8) StreamReader Respstreamreader=NewStreamReader (Respstream,encoding.utf8); Byteread= Respstreamreader.read (Cbuffer,0, the);  while(Byteread! =0) { stringStrresp =New string(Cbuffer,0, Byteread); Strbuff= Strbuff +Strresp; Byteread= Respstreamreader.read (Cbuffer,0, the); } respstream.close (); Txthtml.text=Strbuff;}

C # uses XPath to parse Web pages