Baidu video collection.

In the past, I started to learn how many things Asp.net can write in C. Today I made a small example of Baidu video collection. I want to share with you the experience.
First, analyze the Baidu video search page before writing this program. On Baidu's video search page? Ct = 301989888 & Rn = 20 & Pn = 0 & DB = 0 & s = 0 & FBl = 1024 & word = % BF % B9 % D5 % F0 % be % C8 % D4 % D6 knows the parameter information, we can set parameters by ourselves, get the page we want, and get the page source code extracted using regular expressions. Obviously, the word parameter is the keyword of the search video, and "Earthquake Relief" is the encoded result. In this way, we can create a submission page and submit it by ourselves to obtain the source code of the page after the keyword is searched. We want to do not let users see that we submit data to the Baidu page, so not directly in the form submitted to the http://video.baidu.com/V, so we want to make a page, it is used to process submitted search keywords and obtain connections. The key information on this page is as follows:

String word = This. request ["keyword"];
String url = "http://video.baidu.com/V ";
URL + = "? WORD = "+ word; // link a page

It's easy to write. I will upload the written file to the specific code, and I can test it myself. The above code is to get the source code of the page after receiving the keywords submitted on the previous page and then concatenate a string connecting to the Baidu video. However, if you write in the above way, there will be a problem. Let's imagine, for example, the spell out connection is http://www.video.baidu.com/V? WORD = when the earthquake relief service is connected to this page in Asp.net, Chinese characters are not encoded, so the accessed page will contain garbled characters. Because we can see that after submitting from Baidu, it will change to % BF % B9 % D5 % F0 % be % C8 % D4 % D6 if it is in. the submission in. NET is not the same as the above, because Baidu encoding is gb2312, and Asp.net is UTF-8 by default. So the encoded strings are different, so we need to modify the configuration file, but it is not worth modifying the configuration file with such a small function. Therefore, after receiving the string, we encode it in gb2312 mode.

System. Web. httputility. urlencode ("Earthquake Relief", encoding. getencoding ("gb2312"); // encode the keyword in gb2312 mode, so that it is consistent with Baidu

OK. After encoding, we can obtain the source code and use a regular expression to extract key information.
It is very easy to get the page source code. The following is a method I have written to provide the URL Connection address to return the page source code.

Public String gethtml (string URL)
{
WebClient web = new WebClient ();
Byte [] buffer = web. downloaddata (URL );
Return encoding. Default. getstring (buffer );
}

The next step is to use a regular expression to get the information we want. I will not explain this regular expression, because the regular expression is always written. You can search for documents online to learn more.

RegEx regexvideo = new RegEx ("\\< TD \>\\< Div \\ S + class \\= x \\>\< A \\ S + href = \"(? <Videourl>. +) \ "\ s + onmousedown = \". + \ "\ s + Title = \"(? <Title>. +) \ "\ s + target = \" _ blank \ "> . +) \ "\ s + Alt = \". + \ "\>"); // This is the regular expression that I wrote to retrieve the video information in the source code after searching for the video.

Matchcollection MS = regexvideo. Matches (htmlcode); // get the video information
Foreach (Match m in MS) // traverse the result obtained by the Regular Expression
{
This. response. write (" </img> <br> "); // output the obtained Image
This. response. write ("<a href = \" "+ M. groups ["videourl"] + "\"> "+ M. groups ["title"] + "</a> <br>"); // output the real video address
}

The above is the regular expression for each video information in the source code of the Baidu page.

Let's sum up the idea of designing this program. You can skip this code and clarify the idea. I think the code is not very important, but it is important to think about it.

1. analyze the connection address of the Baidu video search page and analyze the video format of the source code of the page, which is to analyze the HTML element.
2. Spell the connection address, get the source code and retrieve the information.

Once we get the information, we can directly output the information, or we can extend the idea, use winfrom to write a tool, and use multiple threads to automatically collect data and input it into the database. There are still many scalable areas.

Don't laugh when writing it out. Maybe you think it is very simple, but I will always appreciate the achievements of this newbie, And I will comment on it a lot. I can better understand my shortcomings.