. Get/Manipulate Web page HTML code with WebBrowser in NET4.5 WFP

Introduction

Want to give yourself before the Web novel crawler update, has been written in WinForm form of the program, so this update intends to change the UI to WPF (because I heard that WPF is beautiful), and by the way to introduce WPF learning.

Then as a web crawler, the most important thing is to get the HTML source, the usual way to obtain a few: HttpWebRequest, WebRequest, WebClient, WebBrowser and so on. Because I wrote here crawler software for the site need to first login to see the content of the novel, and I crawl the content is not many short stories, so do not need to take into account the speed of crawling, it is natural to use WebBrowser to achieve access to the Web site after landing source. and in order to speed up the crawl speed I consider to open several WebBrowser threads, at the same time on multiple pages to collect, temporarily do not know whether this idea can be realized, this is something. (On the other hand, if you want to quickly crawl a large number of pages, you can consider using the WebClient, if you have login requirements to do a simulated login cookie, but this is quite troublesome)

Problem

After you create a new WPF project, the first thing to do is to drag a WebBrowser control to a window, because I am familiar with web development, so XAML is not difficult for me. However, in the process of obtaining the source code of the Web site, the WebBrowser, WPF properties and methods do not seem to be able to get HTML!

Practice

Ask the search engine first, however, there seems to be no similar demand on the Internet, all WPF in how to use WebBrowser HTML elements, presumably is to operate the DOM through the IHTMLElement interface. But since all the DOM is in operation, it's no problem to get HTML.

1. Add Microsoft.mshtml.dll in References

Find the item you want to reference in

2. Manipulating HTML

Then you can use HTMLDocument to manipulate the HTML, remember to refer to the namespace: mshtml. The content is then pasted in the form of code.

usingMshtml;......webbrowser.navigate ("http://www.baidu.com/");//... Wait for Web page to load ...HTMLDocument Doc= This. wb3. Document asHTMLDocument;//get the HTML code inside the bodystringHTML =Doc.body.innerHTML;//I don't know how to get all the HTML code for the moment.//get elements by ID//get input by IDIHTMLElement Inputsearch= (IHTMLElement) Doc.all.item ("kw");//set the Value property for inputInputsearch.setattribute ("value","Set Search Content");//Get Search buttonIHTMLElement Submitsearch= (IHTMLElement) Doc.all.item ("su");//Click the buttonSubmitsearch.click ();//Get Formmshtml. IHTMLWindow2 window=(MSHTML. IHTMLWINDOW2) Doc.parentwindow;//inject JavaScriptWindow.execscript ("alert (123);","JavaScript");//injection of non-pop windows, etc.Window.execscript ("function alert () {return;}","JavaScript"); Window.execscript ("function confirm () {return;}","JavaScript");//The second alert will not executeWindow.execscript ("alert (456);","JavaScript");

/// <summary>///This method is said to be able to block JS, called in the navigated event, not tested/// </summary>/// <param name= "WB" ></param>/// <param name= "Hide" ></param> Public voidHidescripterrors (WebBrowser WB,BOOLHide) {FieldInfo Ficomwebbrowser=typeof(WebBrowser). GetField ("_axiwebbrowser2", BindingFlags.Instance |bindingflags.nonpublic); if(Ficomwebbrowser = =NULL)return; ObjectObjcomwebbrowser =ficomwebbrowser.getvalue (WB); if(Objcomwebbrowser = =NULL)return; Objcomwebbrowser.gettype (). InvokeMember ("Silent", BindingFlags.SetProperty,NULL, Objcomwebbrowser,New Object[] {Hide});

. Get/Manipulate Web page HTML code with WebBrowser in NET4.5 WFP