Scrapy is a python-only web crawler tool that currently has only python2.x versions.
- Installation
Scrapy need more support cubby, installation is very cumbersome, test directly with Easy_install or PIP installation will automatically download the support library installation needs, but because the network or other reasons always install failure, it is best to install each installation package separately.
- Download Install the Microsoft Visual C + + redistributable Package (Note that the link is 64-bit) http://www.microsoft.com/en-us/download/details.aspx ? id=15336
- Download and install pywin32:http://sourceforge.net/projects/pywin32/files/
- Twisted:http://twistedmatrix.com/trac/wiki/downloads
- Zope.interface zope.interface pypi page download and install the Egg pack with Easy_install
- lxml:http://pypi.python.org/pypi/lxml/
- Pyopenssl:https://launchpad.net/pyopenssl
- Scrapy (link to win AMD64 python2.7 version) scrapy
The above 2--7 can also be installed via Easy_install or PIP.
Python scrapy Crawler Primer