Write your own beautiful picture downloader

Preface : See the title may be people feel familiar, yes, this blog post is based on Yang Zhengko teacher's "Baidu Beautiful picture Downloader development tutorial." NET version ". Because I also watched the tutorial, feel very fun, and then want to self-completed once, as the previous basic Learning content review and use. In the form of blog and everyone to share the whole development process is to take this opportunity to re-organize ideas, exercise their ability to express. If you are familiar with the following knowledge points, you can ignore this article.

I. Major TECHNOLOGIES

• Basic use of WinForm common controls
• HttpWebRequest Request other Site content
• Newtonsoft.Json.dll Component parsing Json data
• Read and write operations for IO file streams
• Multithreading and updating UI controls across threads
• IrisSkin4.dll Components beautify the skin

Two. Demand analysis

(1) First, we open the Baidu image, enter the search keyword "Beauty", then large tracts of beauty blowing. Want to bulk download these "beauty" must find a way to get each picture request address, but Baidu Server will not provide us with specific download links, we can only try to analyze, to test. When dragging the drop-down box we will find a phenomenon, that is, the picture is loaded in real-time display, not when we open the page loaded once. We also have a general understanding that images are loaded by asynchronous requests, which is often called Ajax requests. Through the nature of the phenomenon, we press "F12" into the developer mode, to see what the asynchronous request is going on, and what it will respond to the browser.

　　Action: Press F12, Reload page, and view XHR request , drag the scroll bar. Interface appears:

(2) The two asynchronous requests are generated when the scrollbar is dragged, that is, the request address that is used to asynchronously load the next page of the picture. Next we look at the response content of one of the requests:

　　It was observed that the response content was in JSON format, where we found what we wanted in the IMGs array node. The IMGs array contains an index of 0-59 a total of 60 JSON objects, tested and compared, each object corresponds to a picture information, where Objurl is true for each picture. 60 objects, which means that 60 images are loaded each time the request is counted, one page at a time.

(3) Next we analyze the above two to the Baidu server asynchronous request URL:

1.http://image.baidu.com/search/avatarjson?tn=resultjsonavatarnew&ie=utf-8&word=%e7%be%8e%e5%a5%b3&CG=girl&PN=60&rn=60&ITG=0&z=0&FR=&width=&height=&LM=-1&ic=0&s=0&st=-1&gsm=350700003c2.http://image.baidu.com/search/avatarjson?tn=resultjsonavatarnew&ie=utf-8&word=%e7%be%8e%e5%a5%b3&CG=girl&PN=120&rn=60&ITG=0&z=0&FR=&width=&height=&LM=-1&ic=0&s=0&st=-1&gsm=7107000078

First, they are both get requests and the request addresses are the same. However, there is a difference between the individual parameter values following the address: one is "PN" and the other is "MSG".

"PN" parameter: The value is 60, 120, according to paragraph (2) described, it is speculated that "PN" is the number of images loaded;

"GSM" parameter: Not too understand, according to Baidu results, speculated that "GSM" as a communication standard, as a random value, does not affect the request process;

"word" parameter: "%E7%BE%8E%E5%A5%B3" after decoding through UrlDecode, the content is "beautiful", speculated "word" for the search keyword.

Analysis of this, we can go to the Baidu server to send a request, we ourselves to define the search keywords and request the number of pictures. This is also the most critical step in implementing this development process.

　　Summary: Through the previous three sections of the narrative, we analyzed the beauty picture of the request process and response content. So far, we develop Baidu beautiful picture Downloader has the general direction: the simulation browser to the Baidu server to send a request, parse response Jinjong get the image address, download saved to the local.

Three. Solution 3.1 Interface Setup

Based on the results of the analysis, we got two controllable elements from the request address: The search keyword and the number of images loaded each time . Then we can build a good software interface:

Here is a brief description of the software workflow: When the user enters the search keywords and gets the number of pages, the background opens a new thread request Baidu server, the request succeeds then parses the response JSON in the IMGs array each picture real Objurl, then requests each objurl in turn, It is eventually saved as a file stream in a local folder.

3.2 Core Code Interpretation 3.2.1 Open a new thread (prevent the screen from suspended animation) request a custom address

            stringKeyWord =uri.escapedatastring (Txtkeyword.text); //Get urlencode encoded keywords              intPageCount =Convert.ToInt32 (Numpage.value); //Get the number of request pages             stringURL =string. Format (@"http://image.baidu.com/search/avatarjson?tn=resultjsonavatarnew&ie=utf-8&word={0}&cg=girl& PN={1}&RN=60&ITG=1&Z=0&FR=ALA&LM=-1&IC=0&S=0&ST=-1&GSM=AA0A0000B4", KeyWord, PageCount * -);Thread=NewThread (() ={processdownload (Keyword,url, PageCount);            //Processing request }); Thread. Start ();

3.2.2 HttpWebRequest to the Baidu server to send request processing

        Private voidProcessdownload (stringKeyWord,stringRequrl,intPageCount) {             for(inti =0; i < PageCount; i++) {HttpWebRequest Request=(HttpWebRequest) httpwebrequest.create (Requrl); using(HttpWebResponse response =(HttpWebResponse) request. GetResponse ()) {if(Response. StatusCode = =Httpstatuscode.ok) {#regionSuccessful responseusing(Stream stream =Response. GetResponseStream ()) {using(StreamReader reader =NewStreamReader (Stream)) {                                stringResponsebody =Reader.                                ReadToEnd (); Try                                {downloadpage (responsebody);                                // Download all pictures of the current page  } Catch(Exception ex) {Appendlog (ex).        Message);                            // Print Exception Log  } }                        }                        #endregion                    }                    Else{showmsg ("get the first"+ (1+ i) +"page failed with error message:"+Response.                    StatusCode); }                }            }        }

3.2.3 Parsing Json response content with third-party components Newtonsoft.Json.dll

        Private voidDownloadpage (stringresponsebody) {Jobject Bodyroot=(Jobject) jsonconvert.deserializeobject (responsebody); Jarray Imgsroot= (Jarray) bodyroot["IMGs"];  for(inti =0; i < Imgsroot.count; i++) {jobject img=(Jobject) imgsroot[i]; stringurl = convert.tostring (img["Objurl"]); Try                {downloadimage (URL);                // download pictures by URL } Catch(Exception ex) {Appendlog (ex).                Message); }            }        }

3.2.4 Download image to local

        Private voidDownloadimage (stringImgurl) {            stringSavepath = Path.Combine (Txtsavedir.text, (Path.getfilename (Imgurl)));//set the Save directory for the pictureHttpWebRequest request =(HttpWebRequest) httpwebrequest.create (Imgurl); Request. Referer="http://www.baidu.com/";//forgery of the request is Baidu's own request to deceive the server            using(HttpWebResponse response =(HttpWebResponse) request. GetResponse ()) {if(Response. StatusCode = =Httpstatuscode.ok) {using(Stream stream =Response. GetResponseStream ()) {using(FileStream fswrite =NewFileStream (Savepath, FileMode.Create)) {stream.                        CopyTo (Fswrite); }} prodownload.begininvoke (NewAction (() ={Prodownload.value+=1;                /////Asynchronous update progress bar to resolve cross-thread update control issues  }); }                Else{showmsg ("Picture download failed with error message:"+Response.                StatusCode); }            }        }

Here are only some of the core code, beginners and interested friends please refer to the full code: Click to download

3.3 Skin Beautification

About WinForm form beautification, there are two ways to understand this: the first is to override the control of the WinForm itself, but this requires a very familiar control of the various properties and events and requires a high degree of GDI drawing technology. The second is the use of third-party WinForm skin components. I chose the second way decisively, after all, not enough. This is the use of the "IrisSkin4.dll" component, which is also popular today, to achieve a skin beautification effect.

"IrisSkin4.dll" How to use it? Because our downloader program has only one form, you can use the following simple way:

1, put the IrisSkin4.dll assembly into the project file debug folder, and then add "References" in the project;

2. Add a namespace to the form's post-class: Using Sunisoft.irisskin;

3. Call the Initializeskin method in the constructor of the form class to initialize the component.

        Private void Initializeskin ()        {            new skinengine ();      // initializing a Component Object            " Skin/diamondred.ssk ";   // Loading skin files            true;                 // the skin            is applied to all forms true ;        }

Initializeskin Method

In this way, the beautification work is very simple to complete. Here to provide IrisSkin4.dll to the needs of friends: Click to download

3.4 Effects Demo four. Reflection Summary

First talk about this writing, although the function has been implemented, but there are many extensibility, here we just crawl the keyword for "beautiful" Ajax request address, we can analyze other keyword requests to find some rules, perhaps can be combined with regular expressions to achieve more types of images of bulk download. This is the biggest harvest is to understand the use of the HttpWebRequest class, later used to crawl data from the Internet may be very useful.

Read the teacher Yang's tutorial, but in the process of their own code to stop the pause, follow the idea of writing but there are always some details to take into account. It seems that it is not feasible to study forward, and to take time to review the basis of previous studies.

Write your own beautiful picture downloader