0 Basic Python crawler frame scrapy installation configuration

Source: Internet
Author: User
The previous 10 crawler notes on the ground continue to record some simple Python crawler knowledge,
Used to solve the simple bar download, performance point calculation naturally.
However, in order to bulk download a large number of content, such as all the questions and answers, it is not a bit more than the edge.
As a scrapy, the reptile frame is on the way!
Scrapy = Scrach+python,scrach This word is the meaning of crawling,

Scrapy's official website address: dot me point me.

So here's a simple demonstration of the scrapy installation process.
Specific process reference: http://www.jb51.net/article/48607.htm
Friendly reminder: Be sure to follow the Python version of the download, or the installation will remind you that Python is not found. We recommend that you install 32 bits because some versions of the software must be 64 bit hard to find.

1. Install Python (32-bit recommended)

Recommended installation python2.7.x,3.x seems not yet supported.
After installation remember to configure the environment to add the Python directory and the scripts directory under the Python directory to the System environment variable path.
Enter Python in cmd if the version information indicates that the configuration is complete.

2. Installing lxml

lxml is a library written in Python that allows you to work with XML quickly and flexibly. Click here to select the corresponding Python version to install.

3. Installing Setuptools

To install the egg file, click here to download the corresponding version of Setuptools for python2.7.

4. Installing Zope.interface

You can use the third step download setuptools to install the egg file, now also has the EXE version, click here to download.

5. Installing twisted

Twisted is an event-driven network engine framework implemented in Python, click here to download.

6. Installing Pyopenssl

Pyopenssl is a python-OpenSSL interface, click here to download.

7. Installing Win32py

To provide Win32API, click here to download

8. Installing Scrapy

It's finally time for excitement! After installing so many widgets, it was finally the protagonist's turn.
Enter Easy_install scrapy return directly in CMD.

9. Check the installation

Open a CMD window, execute the scrapy command anywhere, and get the following page to indicate that the environment is configured successfully.

  • Related Article

    Contact Us

    The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

    If you find any instances of plagiarism from the community, please send an email to: info-contact@alibabacloud.com and provide relevant evidence. A staff member will contact you within 5 working days.

    A Free Trial That Lets You Build Big!

    Start building with 50+ products and up to 12 months usage for Elastic Compute Service

    • Sales Support

      1 on 1 presale consultation

    • After-Sales Support

      24/7 Technical Support 6 Free Tickets per Quarter Faster Response

    • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.