Python Crawler Learning--use selenium and phantomjs to crawl JS dynamically loaded Web pages

1. Installing Selenium

Pip Install seleniumcollecting Selenium  downloading SELENIUM-3.4.1-PY2.PY3-NONE-ANY.WHL (931kB)    100% | ████████████████████████████████| 942kB 573kb/s Installing collected packages:seleniumsuccessfully installed selenium-3.4.1

2. Installing PHANTOMJS

: http://phantomjs.org/download.html

The downloaded version is: phantomjs-2.1.1-linux-x86_64.tar.bz2

Unzip the downloaded file and move the folder to the/usr/local directory

sudo mv Phantomjs-2.1.1-linux-x86_64/usr/local/phantomjs

Add under/etc/profile, then Source/etc/profile

Export Phantomjs_home=/usr/local/phantomjsexport path= $PATH: $PHANTOMJS _home/bin

Test whether the installation was successful

phantomjs-v2.1.1

Python Crawler Learning--use selenium and phantomjs to crawl JS dynamically loaded Web pages