C # Implementation of crawl Site page Content instance method _ Practical skills

Grab the news section of Sina, as shown in the picture:

Use Google browser to view the source code: Through the analysis that we are looking for content in the following two tags:

Copy Code code as follows:

<!--publish_helper name= ' Highlights-News ' p_id= ' 1 ' t_id= ' 850 ' d_id= ' 1 '-->

Content....

<!--publish_helper Name= ' News-financial ' p_id= ' t_id= ' d_id= ' 1 '-->

As shown in the figure:

Content....

Use VS to create a Web site as shown in the picture:

We download the network data mainly through the WebClient class to implement.

Use the source code below to get what we choose:

Copy Code code as follows:

protected void Enter_click (object sender, EventArgs e)
{
WebClient we = new WebClient (); Mainly use WebClient class
Byte[] Mydatabuffer;
Mydatabuffer = We.  Downloaddata (Txturl.text); The method returns a byte array, so you need to define a byte[]
string download = Encoding.Default.GetString (Mydatabuffer); Encode the downloaded data

Get news content between two values by querying the source code
int startIndex = download. IndexOf ("<!--publish_helper name= ' Highlights-News ' p_id= ' 1 ' t_id= ' 850 ' d_id= ' 1 '-->");
int endindex = download. IndexOf ("<!--publish_helper name= ' News-financial ' p_id= ' t_id= ' d_id= ' 1 '-->");

string temp = download.  Substring (StartIndex, Endindex-startindex + 1); Intercept news Content

Lblmessage.text = temp;//Display the intercepted news content
}

Effect as shown:

Finally: In addition to saving the downloaded data as text, you can save it as a file type and a stream type.

Copy Code code as follows:

WebClient WC = new WebClient ();
Wc. DownloadFile (TextBox1.Text, @ "F:\test.txt");
Label1.Text = "File Download complete";

Copy Code code as follows:

WebClient WC = new WebClient ();
Stream s = WC. OpenRead (TextBox1.Text);

StreamReader sr = new StreamReader (s);
Label1.Text = Sr. ReadToEnd ();