C # batch download of icons,

C # batch download of icons,

This article is a little long. It took several nights to edit and modify it. If there is a problem with wording and layout, please forgive me. This article is divided into four parts: the main content below is also the basic process of software development.

Phase	Description
Requirement Analysis	It mainly describes the purpose of implementing this program and the analysis of requirements, that is, why does it take time to write and what functions are required;
Solution Design	Design a feasible solution (even if there may still be some problems) based on the existing requirements, such as what the user needs to enter, what the program needs to handle, and the design of databases, functions, and interfaces;
Programming implementation	The batch download function of icons is implemented through. NET programming, focusing on the problems encountered and solutions.
Result Display	Demonstrate and share implementation tools and achievements, and summarize experiences.

 

I. Requirement Analysis

In normal program development, to quickly build a more beautiful user interface, you often need to download some icons as the appearance of buttons, controls, etc, you even need to create some specific icons or images by yourself. I had to say that I needed some technical and aesthetic skills. I downloaded it and found it online, finding a set of icons suitable for themes, colors, sizes, and appearance is really not easy.

Fortunately, there are many websites dedicated to downloading icons on the internet, commonly used:

Http://www.easyicon.net/

Https://www.iconfinder.com/

Http://www.haotu.net/

Http://www.iconpng.com/

Http://findicons.com/

Http://www.flaticon.com/

Http://www.iconspedia.com/

Http://icones.pro/

These websites have their own advantages. In common, they all contain a large number of icons. I personally prefer searching, downloading, and enjoying their websites on EasyIcon. It has the following advantages:

(1) Chinese and English search is supported. EasyIcon supports searching in Chinese and English. Of course, the original icon name is still in English, but before searching, you can use Baidu translation API to translate Chinese into English and then search for it.

(2) good user experience. Many websites need to click buttons such as "next page" during browsing. It supports keyboard shortcuts and has a good experience. its interfaces and texts are also lively, for example, in a heat-based ranking, It is elegantly referred to as "the first shot-up priority ".

(3) keep updated. As code writing, we are most afraid that open-source things will not be updated any more, and the update frequency of EasyIcon icons will be enough.

(4) package and download. Sometimes, we can use more than one icon to package and download it. (However, this function has certain limitations, such as the quantity limit for each package download, and the download size and format cannot be set, this is why we need to re-write a batch download tool .)

Therefore, we need a program to download icons of different formats and sizes to the local device in batch for search and utilization.

Ii. solution design

1. Browser Download icon

The design scheme does not come up directly, but should be analyzed and determined based on the actual situation. We can use a browser to download an icon for a try.

Goals: http://www.easyicon.net/iconsearch/iconset:fatcowhosting-icons/

The URL contains 2000 (40 pages) of different sizes and formats and icons. Fatcowhosting-icons is the classification name of these Icon Sets.

Click the first icon to go to the other details page: http://www.easyicon.net/530832-zoom_selection_icon.html. here we can see a lot of details.

Click the PNG icon to download the image. (This download is the code in the innermost loop of the Code in the future .) We see the real: http://download.easyicon.net/png/530832/32/

Any browser or custom program on which the download URL is available can be downloaded.

2. Analysis

Let's see the address of each page:

Http://www.easyicon.net/iconsearch/iconset:fatcowhosting-icons/1/

Fatcowhosting-icons indicates the name of the icon set, and 1 indicates the number of pages.

Let's analyze this address: http://download.easyicon.net/png/530832/32/

This can be divided into: fixed part + format + icon number + size

Let's take a look at the parameters required for downloading: + file storage path + file name

Comprehensive analysis shows that the icon format, size, and file storage path can be specified by the user. The key isThe icon number and file name are missing..

If we already know the icon number and enter the download URL in the address bar of the browser to submit it, the browser can automatically identify the name of the downloaded file. Why? This indicates that after the user submits the address to the server, the server returns some messages, including the file name. Therefore, some programming method is used (as mentioned later, you do not have to worry about querying ),The file name can be obtained..

Okay, nowThe only missing master is the icon number.. By observing other icons on the website, we can see that these numbers are connected. For example, 530832 is the number of Zoom_Selection_icon, and 530831 is the number of Zoom_Refresh; then, let's look at the fatcowhosting-icons set. Each page is 50 (except the last page ), we cannot obtain all the numbers of the icon set based on the number of each icon and the last icon? The answer is yes.

How can we get the first and last numbers? If we use some technical means to get these two numbers ...... And so on. If you can get these two numbers, why not retrieve them directly? Yes,Some method of Web Page capturing should be able to get all numbers.

3. Draw a simple flowchart

The following is a flowchart drawn by an expert V7.9 using the yitu graphic illustration:

4. Write a simple interface

After analysis for so long, write a simple interface to understand our ideas. (C #)

private string [] FileType; // File format
private int [] FileSize; // File size
private string FilePath; // File save path
private int TotalPages; // Total number of icon pages

// Get the total number of icon pages
private int GetTotalPages (string iconsURL) {}
// Get the number of the current page
private string [] GetIDs (string pageURL) {}

private bool DownICO (string [] fileType, int [] fileSize, int totalPages)
{
    // One layer: traverse each page
    for (int i = 0; i <totalPages; i ++)
    {
        // Get all the numbers of the current page
        string [] strIDs = GetIDs ("PagesURL");
        // Two layers: traverse each number
        for (int j = 0; j <strIDs.Length; j ++)
        {
            // Three layers: traverse each size
            for (int k = 0; k <fileSize.Length; k ++)
            {
                // Four layers: traverse each format
                for (int m = 0; m <filePath.Length; m ++)
                {
                    // Generate download link
                    string downURL = "http://download.easyicon.net/format/number/size/";
                    Down (this.FilePath, downURL);
                    // Other operations ...
                }
            } // 4
        } // 3
    }//2
}//1

// Download every icon
private bool Down (string filePath, string downURL) ()

5. Key Issues

The following is a solution to the key issues used in the Code:

(1) which class or method can be used to download all parameters? The DownloadFile method of System. Net. WebClient.

(2) how to get the total number of pages of the icon? According to the observed webpage, each page has an "icon, which can be viewed by page X". "X indicates the total number of pages. You can capture the webpage string;

(3) how to obtain the numbers of all icons on each page? Of course, it is still through web page capture. For example, you can view the number and name of each icon through the review element.

(4) how to get the download icon name? There are two methods: webpage content capturing and extraction based on the information returned by the Service.

Iii. Programming implementation

Programming is relatively simple. Below are two core functions for webpage operations (the first time a webpage is crawled, I don't know if this is the best)

The first function is to obtain the webpage code through the webpage address.

/// <summary>
/// Get webpage code based on URL
/// </ summary>
/// <param name = "strURL"> URL address </ param>
/// <returns> Web code string </ returns>
public static string GetHtmlString (string strURL)
{
     Uri uri = new Uri (strURL);
     HttpWebRequest request = (HttpWebRequest) WebRequest.Create (uri);
     HttpWebResponse response = (HttpWebResponse) request.GetResponse ();
     Stream stream = response.GetResponseStream ();
     string strHtml = "";
     if (stream! = null)
     {
         StreamReader sr = new StreamReader (stream);
         strHtml = sr.ReadToEnd ();
         sr.Close ();
         stream.Close ();
         response.Close ();
     }
     return strHtml;
}

The second function is to obtain the returned headers information based on the download link of the icon submitted to the server. The information contains the icon name.

/// <summary>
/// Get header information based on URL
/// </ summary>
/// <param name = "URL"> URL address </ param>
/// <returns> headers information list </ returns>
public static Dictionary <string, string> GetHeaders (string URL)
{
     Dictionary <string, string> headerList = new Dictionary <string, string> ();
     WebRequest webRequestObject = HttpWebRequest.Create (URL);
     WebResponse responseObject = webRequestObject.GetResponse ();
     foreach (string headerKey in responseObject.Headers)
     {
         headerList.Add (headerKey, responseObject.Headers [headerKey]);
     }
     responseObject.Close ();
     return headerList;
}

Question 1: Verification Code

Programming is not just a one-stop process. You may encounter problems that you didn't think of before.

The biggest problem encountered is the verification problem. If a large number of downloading icons (up to 166 icons for the first time) are submitted to the server, the verification window is displayed. The following figure shows the results obtained using the webBrowser control.

This is the result of returning PHP from another webpage http://www.easyicon.net/api/captcha/captcha.

Solution: At the beginning, the solution was to capture packets and obtain the submitted links and content, just like other programs asking users to bypass the code. Then I had to bypass the code anyway, it is better to let the user directly see this page (of course, this interface is very rough, in fact, you should get this icon and display it in front of the user ), therefore, the webBrowser control is used. Next, an input is required and then submitted. The input adopts the InputBox in VB, which is more convenient and does not need to be paused, the submit button is obtained using the GetAttribute of HtmlElement and executed using the InvokeMember method.

Question 2: false program death

If there are too many downloads, the program interface will surely be suspended and the user experience will be poor. You need to create a new thread, but pay attention to the control information interaction between the new thread and the main thread.

Solution: The following is a delegate function to add and download the returned message to ListBoxAdv.

delegate void SetValueCallback(ListBoxAdv lstA,string log);
private void SetPropertyValue(ListBoxAdv lstA,string log)
{
    if (lstA.InvokeRequired)
    {
        SetValueCallback d = new SetValueCallback(SetPropertyValue);
        lstA.Invoke(d, new object[] { lstA,log });
    }
    else
    {
        lstA.Items.Add(log);
        lstA.SetSelected(lstA.Items.Count-1,true);
        lstA.SelectedIndex=lstA.Items.Count - 1;
    }
}

Call:

SetPropertyValue (lstAdv, "message ......");

Question 3: download failure

Not all icons can be downloaded normally. Even if the icons are repeatedly downloaded, they are easy to appear. The download result is only 25-byte icons (repeated downloads are also invalid), probably because of the network speed.

Solution: Traverses all 25-byte icons, deletes the icons, and downloads them again (of course, it takes time ).

Iv. Results presentation

Main Interface

Download icon

I tested and downloaded the png 32 icon. There are more than 8000 icons on the local and cloud disks. The file is named by number + name, I can download other required icons from the official website and search for the desired icons by name.

Source code download: http://files.cnblogs.com/files/liweis/EasyDown.rar

Outlook

1. How does the Server check the number of continuous downloading icons on the local machine? Whether it is based on IP or others. If we find out its mechanism, can we skip its detection through some code operation instead of using the verification code?

2. How can I query the name of the icon set through some SQL code? If you can, the entire easyicons is not a problem!