Crawler selenium module, crawler selenium
Selenium Module
IIntroduction
# Installation: selenium + chromedriverpip3 install seleniumdownload chromdriver.exe put it in the scripts directory of the python installation path. Note that the latest version is 2.29, not 2.9 domestic image website address: Login> python3Python 3.6.1 (v3.6.1: 69c0db5, mar 21 2017, 18:41:36) [MSC v.1900 64 bit (AMD64)] on win32Type "help", "copyright", "credits" or "license" for more information. >>> from selenium import webdriver >>> driver = webdriver. chrome () # pop-up browser >>> driver. get ('https: // www.baidu.com ')> driver. page_source # Note: selenium3 default supported webdriver is Firfox, and Firefox needs to install geckodriver download link: https://github.com/mozilla/geckodriver/releasesselenium+chromedriver
Ii. Installation
Selenium was originally an automated testing tool, which was used by crawlers mainly to solve the problem that requests could not directly execute JavaScript code. selenium is essentially driving browsers and completely simulating browser operations, for example, jump, input, click, drop-down, and so on to get the results after web page rendering. multiple browsers can be supported from selenium import webdriverbrowser = webdriver. chrome () browser = webdriver. firefox () browser = webdriver. phantomJS () browser = webdriver. safari () browser = webdriver. edge ()