Recently in the study of the crawler, the use of selenium module, encountered a lot of pits, the purpose of this blog is to summarize the experience of the pits and solutions, so that yimeimei less detours!
The following are all based on the python3.x, Windows platform.
1. Selenium Configuration
We do not want to use the Selenium module when we are doing crawlers, but it is not a Python standard library, so we need to install it ourselves.
So how to install it selenium
, this time to use pip
this tool.
1. FirstWe need to download PIP:
The so-called Pip, python install Packet (Python installation package).
2. ThenWe unzip the downloaded Pip, open the CMD command console, CD to the directory just unzipped:
As you can see, Python is not an internal command, and execution fails because the Python executable path is not added to the Windows environment variable.
3. SettingsEnvironment variables:
In the following order:
environment variable, advanced system settings, properties, computer
Add the path to the Python.exe path
(note that the isolation between the different paths is used ;
)
4. VerificationWhether PIP is installed successfully
Use the following command (before using this command, you also need to add the directory where the PIP resides in the environment variable, the default PIP path is Python.exe the same directory folder Scripts
):
5. InstallationSelenium module
Use the following command:
At this point, the Selenium module is the installation completed
6. ConfigurationSelenium make IT support different browsers
In development, we generally use chrome or Firefox, so I will explain these two sections.
When we installed the Selenium module, ready to use it to encode, suddenly found that there are problems:
This occurs because of a lack of chrome/firefox webdriver drivers.
Driven by:
- Chrome/firefox webdriver GitHub Address
- This address has all the Ie/chrome/firefox webdriver
Unzip the downloaded driver and place it python.exe
in the same directory;
Here, you can try the following code:
# coding = utf-8from time import sleepfrom selenium import webdriverbrowser = webdriver.Firefox()browser.get(r"http://www.baidu.com")browser.find_element_by_id("kw").send_keys("python")browser.find_element_by_id("su").click()sleep(5)browser.quit()
A new Firefox window will open, and the selenium configuration is successful.
2. Pycharm 2017.3 hack and use
Pycharm is to play Python, web development, crawler and so on indispensable tools, its excellent sex Needless to say, who use who know!
1. Download and installPycharm2017
Installation Don't say more!
2. CrackingPycharm
Enter Pycharm, click Help -> Register
Then, go to download a cracked software:
Follow the settings:
After the input is complete, click Active
, everything is OK!
In addition, the cracked professional website address:
3.
Once the registration is complete, we will start our first project:
# coding = utf-8from time import sleepfrom selenium import webdriverbrowser = webdriver.Firefox()browser.get(r"http://www.baidu.com")browser.find_element_by_id("kw").send_keys("python")browser.find_element_by_id("su").click()sleep(5)browser.quit()
During the run, the following error may occur:
Although we installed selenium
, but Pycharm can't find this module (package), what's the matter?
The reason is the problem with the Python interpreter we're using (not using the system default interpreter)
The following modifications can be made:
- Project coding, Project interpreter, File---Settings
Select the Python interpreter you want.
There is no corresponding system Python interpreter in the drop-down list below, we can add it ourselves:
After setting the above, return to the code area to run the above code, everything OK!
At this point, it's done!
Web crawler (i): Configure Selenium, Pycharm (Windows platform)