[Python crawler] 26: Selenium +PHANTOMJS use Pyquery to crawl Smart TV website image information

First, Introduction

This example uses selenium +PHANTOMJS to crawl the information of a smart TV website (http://www.tvhome.com/news/) and enter a given keyword to grab the image information.

Given keywords: numbers; fusion; TV

　　　

　

Second, the website information

　　　　

　

Third, data capture

For the above site information, to crawl

1, first crawl information list

Fetch code: Elements = doc (' div[class= "Main_left fl"] "). Find (' div[class=" content "]"). Find (' ul '). Find (' Li ')

2. Grab the picture

Fetch code: Imgurl = Element (' a '). FIND (' img '). attr (' src ');

Self.down_picture (Imgurl)

Iv. Complete code

 defdown_picture (Self, imgurl):"""download image to local:p Aram imgurl: Image URL"""        #http://img.tvhomeimg.com/uploads/2017/06/23/144910c41de4781ccfe9435e736ef72b.jpg        ifLen (Imgurl) >0:filename="'            ifImgurl.rfind ('/') >0:filename= Imgurl[imgurl.rfind ('/') + 1:] u=Urllib.urlopen (imgurl) data=u.read () strpath= Os.path.dirname (OS.GETCWD ()) +'\picture'with open (Os.path.join (strpath, fileName),'WB') as F:f.write (data)

[Python crawler] 26: Selenium +PHANTOMJS use Pyquery to crawl Smart TV website image information