Python page information fetching

1. Features

In the Python parsing HTML This article has already made a preliminary introduction, and then sit further instructions. Python Crawl page information has the following two features:
　　

• A schema that relies on HTML.

• Small changes can cause a crawl to fail, depending on your coding skills.

2. Crawl Example

First look at the Baidu Video web page source code, roughly browse, select the page element to crawl.
　　
　　
Suppose we are going to extract the div tag ID for the relevant content in Focuscarousellist. First, go to the Python command-line environment and first open the page and read the content using the following code.
　　

>>>
>>> Import Urllib
>>> from BS4 import BeautifulSoup
>>>
>>> Httprespone = Urllib.urlopen ("http://video.baidu.com")
>>>
>>> Httprespone.code
200
>>>

Read the page information into a variable in the HTML: html = httpRespone.read() .
Use BeautifulSoup to parse this page: bs = BeautifulSoup(html,"lxml") .
Look for the DIV tag with id ocuscarousellist: focusList = bs.find(‘div‘,id=‘focusCarouselList‘) .
　　
　　
　　
Find all the hyperlinks in this div in focuslist: allLinks = focusList.find_all(‘a‘) .
Available Alllinks[0] Direct access to the contents of the first link:
　　
　　
If you want to find these hyperlinks with the heading "co-police escort suspects attacked by their associates", use the following code:
　　videoLink1 = bs.find(‘a‘,{‘title‘:‘协警押送嫌犯遭其同伙袭击‘})
　　
　　
videolink1[' href ' can get directly to the address of the link.
　　
Find labels for all pictures: imgLinks = focusList.find_all(‘img‘) .
Get the source address of a picture link: imgLinks[0][‘src‘]
　　

Copyright NOTICE: This article for Bo Master original article, without Bo Master permission not reproduced.

Python page information fetching