Htmlagilitypack--Parsing HTML source code

Recent projects need to fetch from the network data parsing HTML source code, but the regular expression is difficult to write, so the online search found the "Htmlagilitypack" class library, agile development, really efficient and extraordinary.

Take notes here, write down your thoughts, and summarize them by the way.

1, htmlagilitypack using XPath for path search, if you are familiar with XML path search, it will be handy to use.

<?XML version= "1.0" encoding= "Iso-8859-1"?><Bookstore>< Book>  <titleLang= "Eng">Harry Potter</title>  < Price>29.99</ Price></ Book>< Book>  <titleLang= "Eng">Learning XML</title>  < Price>39.95</ Price></ Book></Bookstore>

View Code

/BOOKSTORE/BOOK[1]	Selects the first book element that belongs to a bookstore child element.
/bookstore/book[last ()]	Select the last book element that belongs to the bookstore child element.
/bookstore/book[last ()-1]	Select the second-to-last book element that belongs to the bookstore child element.
/bookstore/book[position () <3]	Select the first two book element that belongs to the child element of the bookstore element.
title[@lang]	Select all the title elements that have properties named Lang.
title[@lang = ' Eng ']	Selects all title elements, and these elements have the lang attribute value of Eng.
/BOOKSTORE/BOOK[PRICE>35.00]	Selects all the book elements of the bookstore element, and the value of the price element must be greater than 35.00.
/bookstore/book[price>35.00]/title	Selects all the title elements of the book element in the bookstore element, and the value of the price element must be greater than 35.00.

2, Htmlagilitypack basic usage:

Most commonly used methods: SelectNodes (XPath), selectSingleNode (XPath),

CreateNode (htmlnode.outerhtml) (this method is usually used in the results of the query is analyzed again, unable to operate from the queried node, query is the operation of the large node tree), general usage

The process is typically to get HTML first, which can load static content through HTMLDocument's load () or loadhtml (), or you can htmlweb the get () or load () method to load the HTML for the URL on the network. (This method I have not tried, do not know whether to load the method on the network)
After getting the instance of HTMLDocument, we can use HTMLDocument's Documentnode property, which is the root node of the whole HTML document, it is also a htmlnode, You can then use the Htmlnode selectnodes () method to return multiple Htmlnode collection Object Htmlnodecollection, or you can take advantage of Htmlnode's selectSingleNode () method returns a single htmlnode.

I get data from the website with WebClient, WebRequest, WebResponse Simple Protocol (HTTP, FTP) requests, basically meet the project requirements:

Common properties: InnerHtml InnerText outerhtml

///<summary>//Stream streaming HTML source </summary>//<param name= "URL" ></param>//<returns></returns> P            Ublic static string Downloadhtmlcode (String Url) {string htmlcode = ""; if (string.            IsNullOrEmpty (URL)) return "";            WebClient client = new WebClient (); Stream stream= Client.            OpenRead (URL);            StreamReader sr = new StreamReader (stream); Htmlcode = Sr.            ReadToEnd (); Sr.            Close (); Stream.            Close (); Client.            Dispose ();        return htmlcode; }

Htmlagilitypack.htmldocument HD = new Htmlagilitypack.htmldocument ();
Loading HTML documents
hd. Loadhtml (downloadhtmlcode ("http://www.baidu.com"));

Here are some of the information that is intercepted from the code

Htmlnode strhtml = null;
foreach (var item in HD. Documentnode.selectnodes ("//*[@id = ' header ']")
{
Brand Analysis
strHTML = Htmlnode.createnode (item. outerHTML);
Project.brand = Strhtml.selectsinglenode ("//a[last ()-1]"). Innertext.replace ("Mobile Phone", ""). Replace ("Phone", "") + "<br/>";
}

3. Delete style, script, script comment

From dense source to find valid source information, delete style, script, script comment, parse get innerhtml, load to generic variable, do two operations (output page, database storage)

  //Remove script tags and content                if(HD. Documentnode.selectnodes ("//script")!=NULL)                {                    foreach(varScriptinchhd. Documentnode.selectnodes ("//script") script.                Remove (); }                            //Remove style labels and content                if(HD. Documentnode.selectnodes ("//style")!=NULL)                {                    foreach(varStyleinchhd. Documentnode.selectnodes ("//style") ) style.                Remove (); }                                //Remove Comment tags and content                if(HD. Documentnode.selectnodes ("//comment ()")!=NULL)                {                    foreach(varCommentinchhd. Documentnode.selectnodes ("//comment ()") ) comment. Remove ();//The new Code}

View Code

Summarize:

Htmlagilitypack tools can be quickly started, but also open source, to meet the needs of parsing HTML source.