Python's directional crawler scrapy

1.Scrapy Introduction

Scrapy,python developed a fast, high-level screen capture and web crawling framework for crawling web sites and extracting structured data from pages. Scrapy can be used for data mining, monitoring and automated testing in a wide range of applications.
The attraction of Scrapy is that it is a framework that anyone can easily modify as needed. It also provides a variety of types of crawler base classes, such as Basespider, Sitemap crawler.

2.Scrapy installation (http://www.cnblogs.com/txw1958/archive/2012/07/12/scrapy_installation_introduce.html)

1) Install LIBXML2:

Official homepage: Http://users.skynet.be/sbi/libxml-python/http://pypi.python.org/pypi/pyOpenSSL

: Http://users.skynet.be/sbi/libxml-python/binaries/libxml2-python-2.7.7.win32-py2.7.exe

2) Install Zope.interface

Official homepage: http://pypi.python.org/pypi/zope.interface/

: Http://pypi.python.org/packages/2.7/z/zope.interface/zope.interface-4.0.1-py2.7-win32.egg

3) install twisted

Official homepage: Http://twistedmatrix.com/trac/wiki/TwistedProject

: Http://pypi.python.org/packages/2.7/T/Twisted/Twisted-12.1.0.win32-py2.7.msi

Input command: Pip install Scrapy

4) Install Pyopenssl

Official homepage: Http://pypi.python.org/pypi/pyOpenSSL

: Http://pypi.python.org/packages/2.7/p/pyOpenSSL/pyOpenSSL-0.13.winxp32-py2.7.msi

5) Install Pywin32

: https://sourceforge.net/projects/pywin32/files/pywin32/

6) Install Scrapy

Official homepage: http://scrapy.org/

: http://pypi.python.org/packages/source/S/Scrapy/Scrapy-0.14.4.tar.gz

Python's directional crawler scrapy