Recently in conscious Python, I have seen the knowledge of the previous article (https://www.zhihu.com/question/20799742), crawling videos on the benefits online ...
Since I started to follow, but the main example is based on python2.x, and I began to learn 3.x, the print usage changed after a lot of modules can not be imported, the novice does not know how to solve.
So, to learn (CHE), I wrote one of the code in C # once Shang. In the case of some delay, the hard disk will be occupied more than 3 G ... Boys and girls, pay attention to the body.
Paste the code below: Deliberately left a few bugs in the code to avoid non-programmers getting on the bus
classProgram {Static voidMain (string[] args) { varBasestring ="http://w*w.46ek.c*m/view/{0}.html"; Regex regex=NewRegex (@"Http://m4.26ts.com/[.0-9-a-za-z]*.mp4"); WebClient WC=NewWebClient (); UINTStartIndex =Readstartindex (); UINTloop =Readlooplen (); for(inti =0; I < lop; i++) { varSuburl =string. Format (basestring, StartIndex +i); WebRequest WReq=System.Net.WebRequest.Create (Suburl)Try{WebResponse Wresp=Wreq.getresponse (); Stream Respstream=Wresp.getresponsestream (); using(StreamReader reader =NewStreamReader (Respstream, Encoding.GetEncoding ("GB18030"))) { varHtmlstring =Reader. ReadToEnd (); Match m=regex. Match (htmlstring); if(m.success) {DownloadFile (WC, M.value,string. Format ("{0}.mp4", StartIndex +i)); } } } Catch(Exception exc) {Console.WriteLine ("Error: {0}", exc. Message); } thread.sleep (5); } } Private Static UINTReadstartindex () { while(true) {Console.Write ("Set Start index:"); stringline =Console.ReadLine (); UINTindex =0; if(Uint32.tryparse (line, outindex)) {Console.WriteLine ("Start index setted:"+index); returnindex; } thread.sleep ( -); } } Private Static UINTReadlooplen () { while(true) {Console.Write ("Set loop len:"); stringline =Console.ReadLine (); UINTindex =0; if(Uint32.tryparse (line, outindex)) {Console.WriteLine ("Loop len setted:"+index); returnindex; } thread.sleep ( -); } } Private Static voidDownloadFile (WebClient WC,stringUrlstringlocalname) {Console.WriteLine ("downloading file {1} to {2}", URL, localname); Wc. DownloadFile (URL, localname); Console.WriteLine ("File {0} download completed!", LocalName); }
My first web crawler C # Edition benefits programmer Chauffeur