Python crawler Frame Scrapy Learning Note 9----Selenium

Selenium is used to automate the testing of Web application. However, it has a huge benefit: it allows us to simulate the operation of a person's browser with Python (not just Python) code.

Required software: python2.7, Firefox 25.0.1 (version not too high), selenium2.44.0 (using pip install Selenium installation)

1. Open Browser, request Baidu homepage, 5 seconds to close the browser

From selenium import webdriverimport timebrower = Webdriver. Firefox () brower.get (' http://www.baidu.com ') time.sleep (5) Brower.close ()

Note that the URL must be preceded by HTTP///

2. Open the browser, request a webpage with dynamic content, wait for JS to complete (10 seconds enough), get the current browser render content source code, close the browser

From selenium import webdriverimport timebrowser = Webdriver. Firefox () browser.get (' http://detail.tmall.com/item.htm?id=12577759834 ') time.sleep () Pagesource = Browser.page_ Sourceprint Pagesourcebrowser.close ()

Python crawler Frame Scrapy Learning Note 9----Selenium