Html/xml processing and regular expressions in C #

HTML Parser
A more convenient HTML parsing package is Htmlagilitypack, which can be installed as shown in Visual Studio.

A simple instance code that uses the package is as follows:

         Public Static BOOL Crawlcategoryreviewinfo(stringCategoryurl) {varRESP = Httputils.getresponsedata (Categoryurl);if(resp = =NULL) {logger. Info ("Failed to request the category page from Suning server!");return false; } HTMLDocument document =NewHTMLDocument (); Document.            Loadhtml (RESP); Htmlnodecollection collection = document. Documentnode.selectnodes ("//div[@id = ' Producttab ']//li[contains (@class, ' item ')]");if(Collection = =NULL|| Collection. Count <1)return false;foreach(Htmlnode prodinchCollection) {if(prod = =NULL|| Prod. attributes["Name"] ==NULL)Continue;stringProdId = Prod. attributes["Name"]. Value;if(Prodid.startswith ("000000000")) ProdId = Prodid.substring (9); Htmlnode Commentnode = prod. selectSingleNode (".//a[contains (@name, ' comment ')]/i");if(Commentnode = =NULL)Continue;intCommentcount =int.                Parse (Commentnode.innertext); Console.WriteLine (ProdId +"\ T"+ Commentcount); }if(collection. Count <int. Parse (configurationmanager.appsettings["Cat_page_item_num"]))return false;return true; }

It is important to note that the Xpath,xpath required to find htmlnode within Htmlnode need to be preceded by ".", as Above ".//a[contains (@name, ' comment ')]/i", Otherwise, you may find that you are looking for a global node.

XML DOM
The XDocument in system space System.Xml.Linq can help parse or output an Xml file.
1) Load parse xml:

            var"image_status.xml");            XDocument doc = XDocument.Load(filePath);            var pics = doc.Descendants("pic");            foreach (varin pics)            {                string url = (string)pic.Element("url");                string imgFile = (string)pic.Element("file");                processedImages.Add(url, imgFile);            }

2) Save build XML

 var  FilePath = Path.Combine (            Path,  "image_status.xml" ); var  docupdate = new  XElement (); foreach             (var  tuple in  processedimages) {var  item = new  XElement (" "image" ); Item. ADD (new  XElement ( "url" , tuple.)                Key)); Item. ADD (new  XElement ( "file" , tuple.)                Value));            Docupdate.add (item); } docupdate.save (FilePath);  

Regular Expression Extraction
The use of regular expressions to extract information, in fact, the logic of different languages are the same, the syntax is slightly different. This is not an introduction, just a sampling example for reference. Note that each matching section starts with "?<->" to name the group, and then the matching data can be used to get the corresponding matching value.

public static void Crawlproductreviewinfo (){String resp = "satisfy ({\"Reviewcounts\":[{\"Onestarcount\": 2,\"Twostarcount\"70A\"Threestarcount\": 23,\"Fourstarcount\": 43,\"Fivestarcount\": 431,\"Againcount\": 4,\"Bestcount\"70A\"Picflagcount\"75A\"TotalCount\": 499,\"Qualitystar\": 4.8}],\"ReturnCode\":\"1\",\"Returnmsg\":\"Number of successful access evaluations\"})"; Regex Revregex = new Regex ("\"TotalCount\":(? <comment>.*?),\"Qualitystar\":(? <score>.*?)}");            MatchCollection mc = revregex.matches (RESP); if (MC. Count > 0){var comment = decimal. Parse (MC[0]. Groups["Comment"].                Value); var score = decimal. Parse (MC[0]. Groups["Score"]. Value);}        }

Html/xml processing and regular expressions in C #