Web crawler (i): Configure Selenium, Pycharm (Windows platform)

Source: Internet
Author: User

Recently in the study of the crawler, the use of selenium module, encountered a lot of pits, the purpose of this blog is to summarize the experience of the pits and solutions, so that yimeimei less detours!

The following are all based on the python3.x, Windows platform.

1. Selenium Configuration

We do not want to use the Selenium module when we are doing crawlers, but it is not a Python standard library, so we need to install it ourselves.
So how to install it selenium , this time to use pip this tool.

1. FirstWe need to download PIP:

The so-called Pip, python install Packet (Python installation package).

  • Pip

2. ThenWe unzip the downloaded Pip, open the CMD command console, CD to the directory just unzipped:

As you can see, Python is not an internal command, and execution fails because the Python executable path is not added to the Windows environment variable.

3. SettingsEnvironment variables:

In the following order:

environment variable, advanced system settings, properties, computer





Add the path to the Python.exe path (note that the isolation between the different paths is used ; )

4. VerificationWhether PIP is installed successfully

Use the following command (before using this command, you also need to add the directory where the PIP resides in the environment variable, the default PIP path is Python.exe the same directory folder Scripts ):

  • PIP List

5. InstallationSelenium module

Use the following command:

  • Pip Install Selenium

At this point, the Selenium module is the installation completed

6. ConfigurationSelenium make IT support different browsers

In development, we generally use chrome or Firefox, so I will explain these two sections.
When we installed the Selenium module, ready to use it to encode, suddenly found that there are problems:


This occurs because of a lack of chrome/firefox webdriver drivers.

Driven by:

  • Chrome/firefox webdriver GitHub Address
  • This address has all the Ie/chrome/firefox webdriver

Unzip the downloaded driver and place it python.exe in the same directory;

Here, you can try the following code:

# coding = utf-8from time import sleepfrom selenium import webdriverbrowser = webdriver.Firefox()browser.get(r"http://www.baidu.com")browser.find_element_by_id("kw").send_keys("python")browser.find_element_by_id("su").click()sleep(5)browser.quit()

A new Firefox window will open, and the selenium configuration is successful.

2. Pycharm 2017.3 hack and use

Pycharm is to play Python, web development, crawler and so on indispensable tools, its excellent sex Needless to say, who use who know!

1. Download and installPycharm2017
  • Pycharm 2017


Installation Don't say more!

2. CrackingPycharm


Enter Pycharm, click Help -> Register

Then, go to download a cracked software:

  • Idea_active_proxy.exe

Follow the settings:

After the input is complete, click Active , everything is OK!

In addition, the cracked professional website address:

  • Pycharm Hack Tutorial
3. Once the registration is complete, we will start our first project:
# coding = utf-8from time import sleepfrom selenium import webdriverbrowser = webdriver.Firefox()browser.get(r"http://www.baidu.com")browser.find_element_by_id("kw").send_keys("python")browser.find_element_by_id("su").click()sleep(5)browser.quit()

During the run, the following error may occur:

Although we installed selenium , but Pycharm can't find this module (package), what's the matter?
The reason is the problem with the Python interpreter we're using (not using the system default interpreter)

The following modifications can be made:

  • Project coding, Project interpreter, File---Settings

Select the Python interpreter you want.

There is no corresponding system Python interpreter in the drop-down list below, we can add it ourselves:


After setting the above, return to the code area to run the above code, everything OK!

At this point, it's done!

Web crawler (i): Configure Selenium, Pycharm (Windows platform)

Related Article

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: info-contact@alibabacloud.com and provide relevant evidence. A staff member will contact you within 5 working days.

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.