1. Historical Weather Enquiry Site-Preparation work

First, Introduction

Always wanted to learn about SQL Server, Oracle database optimization techniques, and some cool HTML reports, but no data. If you build your own data, you feel too troublesome, and the effect is not good. So small think of the crawler, crawling data on the Internet.

Now that you've decided to use crawlers to get the data, let's start thinking about what type of data you choose to crawl. There are novel data, watercress movies, weather data and so on, the final small series selected weather data. Baidu for a few days, browse the weather site, found that the history of the weather query site data compared to small series of sensory, paste the site's interface map below.

Ii. Target website

1. Weather Enquiry Homepage http://lishi.tianqi.com/

2, click to choose a City (Acheng)

3. Click to view April 2017 weather

The final desired crawl data is acheng daily weather data, i.e. date, maximum temperature, minimum temperature, weather, wind direction, wind.

Iii. preparatory work

Since the small series from graduation to the present five years, has been using the development of C # development program. Although C # can also be used as a crawler tool, if there is time, then the small series will share with you a C # asynchronous programming crawler demo, but now also want to professional alignment.

Less gossip, the following is the software I developed.

1. Python 3.6

2. SQL Server

The software can go to http://www.itellyou.cn/download

3, Pycharm

Iv. Summary

Software tools require these three to be able.

Small series of self-study for a period of time python, did not systematically read a Python book. Knowledge is entirely from Baidu and the selfless dedication of bloggers, if there is anything wrong, please do not spray.

In addition, if you have time later will share with you. NET-made crawler tools.

1. Historical Weather Enquiry Site-Preparation work