1.PHANTOMJS Installation
Download: http://phantomjs.org/download.html
Extract:
Tar -jxvf phantomjs-2.1. 1-linux-x86_64.tar.bz2
Rename:
MV /usr/local/phantomjs-2.1. 1-linux-x86_64//USR/LOCAL/PHANTOMJS
Soft connection:
ln
[Email protected]/]# PHANTOMJS
Phantomjs>
2.selenium Installation
PIP installation: Pip install Selenium
Use:
defprocess_request (self, request, spider): Driver=Webdriver. PHANTOMJS ()#Driver = Webdriver. Chrome ()Driver.get (request.url) Body=Driver.page_source Input_first= driver.find_element_by_id ('stockid_') Input_first.clear () Input_first.send_keys ('000150') button= driver.find_element_by_id ('Button') Dataclick=Button.Click ()Print(Dataclick) Body=Driver.page_source#driver.switch_to.frame (' I_nr ') #Print ("Access:", Driver.page_source) returnHtmlresponse (Driver.current_url, Body=body, encoding='Utf-8')
Scrapy crawling data using PHANTOMJS and Selenium