Before the collection process, is single-threaded, a picture list, waiting for one by one to finish .... Wasted a lot of time, just today's home bandwidth upgrade, you can use multi-threaded acquisition .... Overnight to improve the original program. Use multithreading to collect ....
setting Ideas: acquisition Target: http://www.8kmm.com, the list of known URLs (List save), the application of multithreading (thread) to read it, to get the URL can not be repeated (locking lock). Allow Random Collection!
Put on a beautiful woman first to lift the spirits!
Multithreaded Core code:
1 #regionGlobal variables2 //Thread List3List<thread> threadsllist =NewList<thread>();4 //URL list5list<string> uurls =Newlist<string>();6 //finished list7list<string> okurls =Newlist<string>();8 //number of beautiful pictures successfully obtained9 Public intImgcount =0;Ten #endregion One A - //take a diagram Start button event - Private voidLbtngetwebimgstart_linkclicked (Objectsender, LinkLabelLinkClickedEventArgs e) the { -lbtngetwebimgstart.enabled =false; - getwebsiteimg (); - } + - //Drawing Stop button event + Private voidLbtngetwebimgstop_linkclicked (Objectsender, LinkLabelLinkClickedEventArgs e) A { at Try - { - foreach(Thread tinchthreadsllist) - { - if(t! =NULL) - { in T.abort (); - } to } + } - Catch(Exception ex) the { *Writelog ("Stop failed:"+Ex. Message); $ }Panax Notoginseng finally - { thelbtngetwebimgstart.enabled =true; + } A } the + /// <summary> - ///Get Picture Main method $ /// </summary> $ Private voidgetwebsiteimg () - { - Try the { -Imgcount =0;Wuyi okurls.clear (); the uurls.clear (); -Threadsllist.clear ();//First initialize the above Wu string[] URLs = TxtUrl.Text.Split (New string[] {"\ r \ n"}, Stringsplitoptions.removeemptyentries); -Lblc.text =URLs. Length.tostring (CultureInfo.InvariantCulture); AboutUurls =Newlist<string>(URLs); $ for(inti =0; I <int. Parse (Numthreadud.text); i++)//Looping Create Threads - { -Thread T =NewThread (Task); -T.name =i.tostring (); AT.isbackground =true; + Threadsllist.add (t); the T.start (); - } $ } the Catch(Exception ex) the { the Writelog (ex. Message); the } - } in the the Public voidTask () About { the while(Uurls.count >0) the { theLblmsg.text =string. Format ("remaining: {0}, accessed: {1}, current number of threads: {2}", Uurls.count, (Okurls.count +1), threadsllist.count); + stringURL =GetUrl (); - if(url = ="") the {Bayi Break; the } the Else - { - if(gethttpimg (URL) = = $)//only state 200 is normal, gethttpimg is my method of encapsulation, get the Web page, and get all the canonical pictures. the { the okurls.add (URL); theLblinfostart.text = (int. Parse (Lblinfostart.text) +1). ToString (CultureInfo.InvariantCulture); the writehtml (txturl.text); - if(Lblc.text = = Lblinfostart.text)//determines whether the current quantity is total. the { theLblmsg.text ="Complete"; the }94 } the } the } the }98 About - /// <summary>101 ///Thread loads lock to prevent multiple threads from removing the first one from the list at the same time102 /// </summary>103 /// <returns></returns>104 Public stringGetUrl () the {106 Lock("GETURL")107 {108 if(Uurls.count >0)109 { the stringURL = uurls[0];111Uurls.removeat (0); the returnURL;113 } the Else the { the return "";117 }118 }119}
Code super simple, test crawl a few lists, no problem, and so will use it to catch more mm ...
WinForm Collection website Beautiful picture program---multi-threaded article