C #-collect Baidu posts

Http://icuit.cn/bak/Release.rar

The basic principle is to request the HTML source code and parse the HTML Tag.

First, use the system. net. webresponse and system. net. webrequest methods to obtain the HTML string through the URL.

Then parse the HTML Tag. It is said that regular expressions are the best practice.

However, due to lack of learning skills, a third-party library htmlagilitypack can be used to obtain nodes like XML.

Htmlagilitypack includes getelementbyid, selectsinglenode, selectnodes, getattributevalue, and other methods,

Innerhtml, innertext and other main attributes, coupled with some complex string analysis, can basically meet the requirements.

SQLite is still used for data storage. However, it seems that concurrency is normal and is often locked.
Therefore, when 50 threads are enabled to fetch data, they still have to lock the Insert Process.