. Net implementation (webbrowser data collection-Final)

Let's continue with the previous article ". net implementation (webbrowser data collection-Basic) ", due to the time relationship, this article was not written in time; after the previous Article was published, some bloggers have come to disagree and feel that such an article is meaningless. The efficiency of webbrowser data collection is low and the efficiency of webrequest can be improved. I do not understand it, why is the efficiency improved when HTTP is used for data collection? In the collection process, there are also various high-level protocols that need to be switched to the underlying protocol, I personally feel that webrequest is more extensible. My webbrowser data collection is not about data capture efficiency, but focuses on the principles of the webbrowser control, you can use webbrowser to easily integrate HTML Web pages. My next article ". net implementation (webbrowser data collection-continued) ", it will use webbrowser for hybrid use with HTML web pages, in the HTML object, I want to read the database in my webbrowser control and present the winform control in HTML, then, we dynamically enter the data in our winform into the HTML webpage. This kind of humanization, convenience, and simulation can't be replaced by webrequest, most of our software is provided to users, and a friendly user interface is required. [Wang qingpei is copyrighted, for more information, see

Today, I want to talk about how to capture data through webbrowser. The previous article did not talk about how to capture data, but briefly explained the origin of webbrowser controls and some interoperability things, this article will fully explain the problems encountered when using webbrowser to capture data. I have been capturing data for a long time, the first time I captured data, I worked with a good partner of mine. At that time, I encountered a lot of difficulties and solved them silently. After the project was completed, I decided to share this experience with you. I feel that the biggest problem during Web browser crawling is webpage loading, one event in the webbrowser control is the documentcompleted event that we must use. After loading the webpage, we need to determine whether the current page is a logon page, the web page capturing tool we made cannot be a dumb-like manual crawling. We are completely automated. We only need one login, currently, verification codes are basically required for webpage login. Therefore, we need to manually identify and enter the verification code for Logon. Once the logon is successful, everything is automatically completed by the system. For example: the process of dynamic jump to the Collection page, data capture, page flip, and so on has been automated, because the webpage is inevitably not stable in the Process of frequent requests, it will stop for no reason, for example, network disconnection and asynchronous loading are slow. In this way, we can only identify what happened through the Viewing Interface and continue to capture through the breakpoint; we do not discuss whether capturing is better or not to discuss which is more suitable for users' project needs. We need to look at a picture to understand the general principle of capturing:[Wang qingpei has all rights reserved. For more information, please sign it.]

1: this is an integral picture. As the picture is too large, I cut it into three small pictures;

Through detailed steps, we clearly guide how to do it; we switchCodeFragment:

2:

This is my project structure. In the common folder, there is a conspicuous file gatherwebbrowser. CS: this is a simple encapsulation of the webbrowser control. Some event parameters are added to it. Under this file, there is a gatherwebbrowserargs. CS file, which is the delivery object of the webbrowserdocumentcompletedeventargs object;

3:

The purpose of this object is to separate the documentcompleted event so that we can easily process the n-layer jump. Because there are many codes, let's look at several key points, this will help you clarify a clue;

4:

This is our winform window. The webbrowser control is the one I encapsulated. Please note that when we jump to the final collection page layer by layer, We need to disconnect the first event in the event chain, the reason for disconnection is that the documentcompleted event of our webbrowser control is always triggered after the webpage is loaded. Once we enter the collection lifecycle, we do not need to deal with this part of the logic. Here we will involve a technical details "delegate chain". If you do not know much about this, please refer to my ". net short talk about the delegated chain;

Because the data collection logic is complex, especially automatic data collection, many logic problems need to be dealt with. These things are different because of different requirements, so I can only explain this layer, the general implementation method is like this. As long as we analyze the relationship between the front and back, the HTML organization, I believe we can write a good data collection; in the following article, I will use webbrowser to implement a system that interacts with webpages. We often encounter this requirement. For example, we have a C/S system, however, our C/S system depends on the B/S system. We need to use the data of our C/S system as the operating conditions of the B/S system, when entering the query conditions, we need to call our winform window to query the data, and then fill in our winform results in the HTML text box. Please follow up, thank you;