Python Development Simple Crawler (ii)---Crawl Baidu wikipedia page data

First, the development of crawler steps

1. Determine the target crawl strategy:

Open the target page and use the right-click Review element to determine the URL format, data format, and page encoding of the Web page.

① first look at the format of the URL, F12 observe the form of the link;
② look at the target Text information label format, such as text data for div class= "xxx",
③ easy to see encoded as Utf-8

2. Analysis Objectives
Target: Baidu Encyclopedia Python entry
Entry page: Http://baike.baidu.com/item/Python
Entry page URL format:/item/****
Data format:
Title: <dd class= "Lemmawgt-lemmatitle-title" >

Introduction: <div class = "Lemma-summary" >****</div>
Page ID: utf-8

3. Instance Code

Crawl the Baidu Encyclopedia Python Entry and related 1000 page data

Python Development Simple Crawler (ii)---Crawl Baidu wikipedia page data