Go Language parsing HTML

This is a creation in Article, where the information may have evolved or changed.

There are two good libraries:

Https://github.com/PuerkitoBio/goquery

One is

Http://code.google.com/p/go.net/html

HTML is the parser of HTML, parsing HTML text, goquery based on the HTML package, based on the combination of Cascadia package (a CSS selector tool), similar to the function of jquery, the operation of HTML is very convenient.

Use Goquery to find, select the appropriate HTML node, but if you want to modify the selected node, delete the operation, you also need to drill down to use the HTML package.

HTML packages parse HTML text into a tree, which has a lot of node components, and the core of the operation is the operation of node.

Use a few examples to illustrate it:

Doc, Err: = Goquery. NewDocument ("http://sports.sina.com.cn")

Generate a Goquery doc.

The most common use of goquery is the Find function, which resembles the $ () of jquery, and the DOM structure can be selected.

Example 1:

Dhead: = Doc. Find ("Head") Dcharset: = Dhead. Find ("meta[http-equiv]") CharSet, _: = Dcharset. Attr ("content")

This example is used to find out the charset of the page.

Example 2:

Logo: = doc. Find ("#retina_logo")

This is based on the ID in the HTML to select the DOM

Example 3:

Bread: = doc. Find ("Div.blkbreadcrumblink")

Select the class Blkbreadcrumblink div in doc

Example 4:

var faceimg stringvar innerimg = []string{}dom_body. Find ("Div.img_wrapper"). Each (func (i int, S *goquery. Selection) {Imgpath, exists: = S.find ("img"). Attr ("src") if!exists {return}if i = = 0 {faceimg = imgpath}innerimg = Append (innerimg, Imgpath)})

Find all div with class Img_wrapper, then search for img under each div to get img src

Example 5:

Dom_node: = Doc. Find ("[bosszone= ' Zttopic ']"). Find ("a")

This is based on the attribute/value to find the corresponding element

If you want to edit HTML, you need to use HTML. Node, which provides a code to clean up the div, using recursion:

 func clear_dom (pn *html. Node, isgb2312 bool) error {var err errorfor nd: = pn. FirstChild; nd! = nil; {switch nd. Type {case HTML. Elementnode:tn: = Strings. ToLower (nd. Data)//fmt. Printf ("element node:%s\n", TN) if tn = = "Script" | | TN = = "Style" {//delete the elementtmp: = NDND = tmp. Nextsiblingpn.removechild (TMP)} else if tn = = "A" {tmp: = NDND = nd. Nextsiblingif err = convert_dom (tmp, isgb2312); Err! = Nil {return Err}} else if tn = = "span" {tmp: = NDND = nd. Nextsiblingclear_dom (TMP, isgb2312)} else {tmp: = NDND = nd. Nextsiblingif err = convert_dom (tmp, isgb2312); Err! = Nil {return err}}case html.CommentNode:tmp: = NDND = tmp. Nextsiblingpn.removechild (TMP) case HTML. Textnode:tmp: = Ndnd = nd. Nextsiblingif err = convert_dom (tmp, isgb2312); Err! = Nil {return err}default:nd = nd. Nextsibling}}return Nil} 

Where Conver_dom is the text of node nodes transcoding operation, if not required, can be ignored.

Func nodehtml (n *html. Node) string {var buf = bytes. Newbuffer ([]byte{}) HTML. Render (BUF, N) return BUF. String ()}func nodetext (node *html. Node) string {if node. Type = = html. Textnode {//Keep newlines and spaces, like Jqueryreturn node. Data} else if node. FirstChild! = Nil {var buf bytes. Bufferfor c: = node. FirstChild; c! = Nil; c = c.nextsibling {buf. WriteString (Nodetext (c))}return buf. String ()}return ""}

The above two functions, respectively, get the HTML code of the node and the text code. The difference between the HTML code and the text code is that the HTML code is the HTML code that is intact, and the text code shows only the contents of the HTML code, such as an HTML: example, whose text code is "example"