An introduction to the HTML parsing library written by C + +

QT has been learning for some time, a while ago to write a QT version of the blog Park desktop side. At this stage the main want to achieve:

(1) The entries for the recommended articles (including the author's image, the number of views of the blog, the time of publication, etc.) are shown on the blog home page.

(2) You can log in to your own blog Park account and display your own blog post.

(3) When you click the blog post entry that appears, you can display the contents of the blog in a pop-up window.

The automatic refresh function is also implemented.

However, in the process of writing, you need to deal with the HTML document returned by reply, you need a library that can parse the HTML document. I searched the internet to find the C + + written HTML parsing library has htmlcxx, and Google's Gumbo-parser, there are several other libraries, the use of more is htmlcxx and Gumbo-parser. But in the actual use, I found that although the htmlcxx is very good, but it is inconvenient to use, can not customize the same as JS document.getElementById () and document.getelementbyname () such functions , it's also difficult to find HTML tags (maybe I don't have deep research, personal feelings), and Gumbo-parser is under Linux and UNIX, Because of the use of several functions on Windows not provided in the String.h header file, so based on this situation, I wrote an HTML document parsing library. After a lot of pages of testing, the analysis of the more accurate, at least I use the blog home page of the HTML source parsing through their own get_element_by_id function can be correctly obtained to each label node.

After completing this desktop, I will contribute the source code, but also give some examples of use. Hope to help, the library will continue to expand in the future, the implementation of CSS selectors to choose the function of the tag elements, so that we can more convenient to use.

Now no nonsense, directly on the last small case.

First of all, we need to prepare the blog home of the HTML document (in the browser page of the message to select the source code can be obtained, and then the assignment paste saved to a TXT file).

Then, analyze the blog in the middle section of the blog article recommended entries, in the structure of the HTML. You can see that all 20 entries are contained in a id= ' post_list ' div, and each entry is contained in the div of class= ' post_item '.

<DivID= "Post_list">    <Divclass= "Post_item">        <Divclass= "Digg">            <Divclass= "Diggit"onclick= "Diggpost (' Wenjianmuran ', 5049966,256767,1)">             <spanclass= "Diggnum"ID= "digg_count_5049966">0</span>        </Div>        <Divclass= "Clear"></Div>        <DivID= "digg_tip_5049966"class= "Digg_tip"></Div>        </Div>              <Divclass= "Post_item_body">        <H3>                <aclass= "Titlelnk"href= "http://www.cnblogs.com/wenjianmuran/p/5049966.html"Target= "_blank">Python crawler starter Case: Get a list of word-chopping words</a>            </H3>                               <Pclass= "Post_item_summary">                <ahref= "http://www.cnblogs.com/wenjianmuran/"Target= "_blank"><imgwidth= "$"Height= "$"class= "PFS"src= "Http://pic.cnblogs.com/face/728333/20151201230814.png"alt=""/></a>hundred Word slash is a very good word memory app, in the course of learning, it will record every word you learn and the number of times you answer the wrong, through this list can be easily found in the memory of which words are always repeated error cannot remember. We use Python to crawl this information and learn the basics of Python crawlers. First came to the hundred word Slash website: Http://www.baicizhan.com/logi ...</P>                      <Divclass= "Post_item_foot">                            <ahref= "http://www.cnblogs.com/wenjianmuran/"class= "LightBlue">The Paper sword is numb</a>posted on 2015-12-16 00:37<spanclass= "Article_comment">            <ahref= "Http://www.cnblogs.com/wenjianmuran/p/5049966.html#commentform"title=""class= "Gray">Comments (0)</a>        </span>        <spanclass= "Article_view">            <ahref= "http://www.cnblogs.com/wenjianmuran/p/5049966.html"class= "Gray">Read (20)</a>        </span>        </Div>    </Div>    <Divclass= "Clear"></Div>    </Div>                                   </Div> 

Next, you need to parse and get the title of these entries and output them.

#ifndef Html_#defineHtml_#include<Windows.h>#include<string>#include<iostream>std::stringhtml;voidinit_html () {HANDLE hfile= CreateFile (TEXT ("./cnblog.txt"), Generic_read,0, NULL, open_existing, file_attribute_normal, NULL); if(hfile! =Invalid_handle_value) {        intSize =GetFileSize (hfile, NULL); Std::cout<<"File Size:"<< size <<Std::endl; Char* p =New Char[Size +1];        DWORD dwreads; ReadFile (hfile, p, size,&dwreads,null); P[dwreads]=0;        CloseHandle (hfile);        AllocConsole ();        Html.assign (P); Delete[] p; }Else{std::cout<<"can not open file"<<Std::endl; }}#endif

Above is a function to read the contents of an HTML document. (Note that you need to save the file in ANSI format when you read it in this way, otherwise it will appear garbled in Chinese)

#include"Htmlparser.h"#include"parentnode.h"#include"html.h"using namespacestd;intMain () {init_html ();    Hunter::htmlparser parser;    Parser.parser_html (Html.c_str ()); Std::cout<<"finished parser ..."<<Std::endl; Node* node = parser.get_element_by_id (std::string("post_list")); if(Parser.error (). empty ()) {if(node = =0) {Std::cout<<"Can ' t find this node ..."<<Std::endl; }        Else{std::vector<node *> childs = Parser.get_childs_by_class (Node,"Div","class","Post_item");  for(inti =0; I<childs.size (); i++) {Node* Divnode =childs.at (i); Std::vector<node *> divchilds = Parser.get_childs_by_class (Divnode,"Div","class","Post_item_body"); if(Divchilds.size () >0) {Node* Divchild = divchilds.at (0); Node* H3 = Parser.get_child_tag (Divchild,"H3"); if(H3! =0) {Node* A = Parser.get_child_tag (H3,"a"); Std::cout<< Parser.get_tag_text (a) <<Std::endl; }                }            }        }    }    Else{std::cout<< Parser.error () <<Std::endl; } std::cin.Get(); return 0;}

Parsing HTML, using several functions parser.get_element_by_id (std::string ("Post_list")), Parser.get_childs_by_class (node, "div", " Class "," Post_item "), Parser.get_child_tag (Divchild," H3 "), and Parser.get_tag_text (a). These functions are used to get the corresponding label nodes, so let's look at the results of the output below:

See, is not parse and successfully get to the first 20 items of information.

An introduction to the HTML parsing library written by C + +