I wrote a Baidu Network Disk resource search program and Baidu network resource search program.

I wrote a Baidu Network Disk resource search program and Baidu network resource search program.

I. Introduction

The old saying goes with the cloud: "Old and old ". Express a learning enterprising spirit of "more than life, more than learning", is an optimistic encouragement! As a programmer, it should be more like this, otherwise it will be eliminated. What I want to talk about today is not how to learn, but how to find the resources we need online.

 

Ii. Baidu online storage Search Method

There are many ways to find resources, such as Baidu and Google search. I am talking about how to search for resources in "Baidu online storage.

Specific Method: Search for the site: pan.baidu.com keyword in the search engine, as shown in. Google can test Bing.

 

 

3. Write a program

For your own search convenience, simply write a program to crawl Bing search results, such. Demo address: http: // 139.129.12.162/ResourcesCrawl

 

Iv. Program Introduction

1. Use the packet capture tool to analyze the url searched by Bing, for example. We can finally get: http://cn.bing.com/search? Q = keywords & first = the beginning

2. To this url (http://cn.bing.com/search? Q = site: pan.baidu.com keyword & first = the first few times) initiate an Http request (GET method) to obtain html, parse the source code through regular expressions and xPath, and extract information.

3. Code overview.

* DotNet. Http. Core: a dedicated library for Http requests.

* Resouces. Crawl. Common: Some. Net extension methods.

* Resource. Crawl. Services: the library that parses the Html source code and returns information.

* Resource. Crawl. Web: Display layer.

 

V. Download source code (Visual Studio 2013)

Baidu online storage search program