0 Basic Writing Python crawler crawler framework scrapy installation configuration _python

The first 10 crawler notes have continued to record some simple Python crawler knowledge,
Used to solve the simple paste download, the performance point of the calculation of natural.
But if you want to bulk download a lot of content, such as all the questions and answers, it seems to be a bit more than a point.
As a scrapy, the reptile frame is just like this!
Scrapy = Scrach+python,scrach This word is grasping meaning,

Scrapy's official website address: dot i dot me.

So here's a simple demo of the Scrapy installation process.
Specific process reference: http://www.jb51.net/article/48607.htm
Friendly reminder: Be sure to download the Python version, or install it to remind you that you can't find Python. We recommend that you install the 32-bit because some versions of the prerequisite software 64 is not easy to find.

1. Install Python (recommended 32 bits)

Recommended installation python2.7.x,3.x seemingly not yet supported.
After installation remember to configure the environment to add the Python directory and the scripts directory in the Python directory to the path of the system environment variable.
Enter Python in cmd if the version information description is configured to complete.

2. Install lxml

lxml is a library written in Python that allows you to work with XML quickly and flexibly. Click here to select the appropriate Python version of the installation.

3. Install Setuptools

To install the egg file, click here to download the python2.7 version of Setuptools.

4. Install Zope.interface

You can use the third step to download the Setuptools to install the egg file, now there are EXE version, click here to download.

5. Install Twisted

Twisted is an event-driven network engine framework implemented in Python, and click here to download it.

6. Install Pyopenssl

Pyopenssl is the Python OpenSSL interface, click here to download.

7. Install Win32py

Provide Win32API, click here to download

8. Install Scrapy

Finally, it's a very exciting time! With so many widgets installed, the main character is finally on its turn.
Directly in the CMD input easy_install scrapy carriage return.

9. Check Installation

Open a CMD window, execute the scrapy command at any location, and get the following page to indicate that the environment configuration was successful.