Scrapy installation-Windows

Source: Internet
Author: User
I. scrapy Introduction

Scrapy is a fast high-level screen scraping and web crawler framework, used to crawl websites and extract structured data from their pages. it can be used for a wide range of purposes, from data mining to monitoring and automatic testing.

Official homepage: http://www.scrapy.org/

Ii. Install python2.7

Official homepage: http://www.python.org/

: Http://www.python.org/ftp/python/2.7.3/python-2.7.3.msi

1)Install Python

Installation Directory: D: \ python27

2)Add Environment Variables

Slightly system properties-> advanced-> environment variables-> system variables-> path-> Edit

3)Verify Environment Variables

T: \> set pathpath = c: \ windows \ system32; C: \ WINDOWS; c: \ windows \ system32 \ WBEM; D: \ rational \ common; D: \ rational \ clearcase \ bin; D: \ python27; D: \ python27 \ scriptspathext =. com ;. EXE ;. BAT ;. CMD ;. vbs ;. VBE ;. JS ;. JSE ;. WSF ;. wsh

4)Verify Python

 
T: \> pythonpython 2.7.3 (default, Apr 10 2012, 23:31:26) [MSC v.1500 32 bit (Intel)] on win32type "help", "Copyright ", "Credits" or "License" for more information. >>> exit () T: \>

3. Install twisted

Twisted is an event-driven networking engine written in Python and licensed under the Open Source

1)Install setuptools

Download, build, install, upgrade, and uninstall Python packages -- easily!

Official homepage: http://pypi.python.org/pypi/setuptools

: Http://pypi.python.org/packages/2.7/s/setuptools/setuptools-0.6c11.win32-py2.7.exe

Installation Process: omitted

2)Install Zope. Interface

Official homepage: http://pypi.python.org/pypi/zope.interface/

: Http://pypi.python.org/packages/2.7/z/zope.interface/zope.interface-4.0.1-py2.7-win32.egg

Installation Process:

 
T: \> D: \> Cd D: \ python27 \ scriptsd: \ python27 \ scripts> easy_install.exe Zope. interface-4.0.1-py2.7-win32.eggProcessing Zope. interface-4.0.1-py2.7-win32.eggcreating D: \ python27 \ Lib \ Site-packages \ Zope. interface-4.0.1-py2.7-win32.eggExtracting Zope. interface-4.0.1-py2.7-win32.egg to D: \ python27 \ Lib \ Site-packagesadding Zope. interface 4.0.1 to easy-install.pth fileinstalled D: \ python27 \ Lib \ Site-packages \ Zope. interface-4.0.1-py2.7-win32.eggProcessing dependencies for Zope. interface = 4.0.1finished processing dependencies for Zope. interface = 4.0.1d: \ python27 \ scripts>

Verify installation:

D: \ python27 \ scripts> pythonpython 2.7.3 (default, Apr 10 2012, 23:31:26) [MSC v.1500 32 bit (Intel)] on win32type "help", "Copyright ", "Credits" or "License" for more information. >>> import Zope. interface >>>

3)Install twisted

Official homepage: http://twistedmatrix.com/trac/wiki/TwistedProject

: Http://pypi.python.org/packages/2.7/T/Twisted/Twisted-12.1.0.win32-py2.7.msi

Installation Process: omitted

4. Install w3lib

Official homepage: http://pypi.python.org/pypi/w3lib

: Http://pypi.python.org/packages/source/w/w3lib/w3lib-1.2.tar.gz

Decompression process: omitted

Installation Process:

T: \ w3lib-1.2> Python setup. PY installrunning buildrunning build_pycreating buildcreating build \ libcreating build \ Lib \ w3libcopying w3lib \ encoding. py-> build \ Lib \ w3libcopying w3lib \ form. py-> build \ Lib \ w3libcopying w3lib \ HTML. py-> build \ Lib \ w3libcopying w3lib \ HTTP. py-> build \ Lib \ w3libcopying w3lib \ URL. py-> build \ Lib \ w3libcopying w3lib \ util. py-> build \ Lib \ w3libcopying w3lib \__ init __. py-> build \ Lib \ w3librunning install_libcreating D: \ python27 \ Lib \ Site-packages \ w3libcopying build \ Lib \ w3lib \ encoding. py-> D: \ python27 \ Lib \ Site-packages \ w3libcopying build \ Lib \ w3lib \ form. py-> D: \ python27 \ Lib \ Site-packages \ w3libcopying build \ Lib \ w3lib \ HTML. py-> D: \ python27 \ Lib \ Site-packages \ w3libcopying build \ Lib \ w3lib \ HTTP. py-> D: \ python27 \ Lib \ Site-packages \ w3libcopying build \ Lib \ w3lib \ URL. py-> D: \ python27 \ Lib \ Site-packages \ w3libcopying build \ Lib \ w3lib \ util. py-> D: \ python27 \ Lib \ Site-packages \ w3libcopying build \ Lib \ w3lib \__ init __. py-> D: \ python27 \ Lib \ Site-packages \ w3libbyte-compiling D: \ python27 \ Lib \ Site-packages \ w3lib \ encoding. PY to encoding. pycbyte-compiling D: \ python27 \ Lib \ Site-packages \ w3lib \ form. PY to form. pycbyte-compiling D: \ python27 \ Lib \ Site-packages \ w3lib \ HTML. PY to HTML. pycbyte-compiling D: \ python27 \ Lib \ Site-packages \ w3lib \ HTTP. PY to HTTP. pycbyte-compiling D: \ python27 \ Lib \ Site-packages \ w3lib \ URL. PY to URL. pycbyte-compiling D: \ python27 \ Lib \ Site-packages \ w3lib \ util. PY to util. pycbyte-compiling D: \ python27 \ Lib \ Site-packages \ w3lib \__ init __. PY to _ init __. pycrunning install_egg_infowriting D: \ python27 \ Lib \ Site-packages \ w3lib-1.2-py2.7.egg-infoT: \ w3lib-1.2>

Verify installation:

 
T: \> pythonpython 2.7.3 (default, Apr 10 2012, 23:31:26) [MSC v.1500 32 bit (Intel)] on win32type "help", "Copyright ", "Credits" or "License" for more information. >>> import w3lib >>>

5. Install libxml2

Official homepage: http://users.skynet.be/sbi/libxml-python/http://pypi.python.org/pypi/pyOpenSSL

: Http://users.skynet.be/sbi/libxml-python/binaries/libxml2-python-2.7.7.win32-py2.7.exe

Installation Process: omitted

Verify installation:

 
T: \> pythonpython 2.7.3 (default, Apr 10 2012, 23:31:26) [MSC v.1500 32 bit (Intel)] on win32type "help", "Copyright ", "Credits" or "License" for more information. >>> import libxml2 >>>

6. Install pyopenssl

Official homepage: http://pypi.python.org/pypi/pyOpenSSL

: Http://pypi.python.org/packages/2.7/p/pyOpenSSL/pyOpenSSL-0.13.winxp32-py2.7.msi

Installation Process: omitted

Verify installation:

 
T: \> pythonpython 2.7.3 (default, Apr 10 2012, 23:31:26) [MSC v.1500 32 bit (Intel)] on win32type "help", "Copyright ", "Credits" or "License" for more information. >>> import OpenSSL >>>

7. Install scrapy

Official homepage: http://scrapy.org/

: Http://pypi.python.org/packages/source/S/Scrapy/Scrapy-0.14.4.tar.gz

Decompression process: omitted

Installation Process:

 
T: \ Scrapy-0.14.4> Python setup. py install ...... Installing easy_install-2.7-script.py script to D: \ python27 \ scriptsinstalling easy_install-2.7.exe script to D: \ python27 \ scriptsinstalling easy_install-2.7.exe.manifest script to D: \ python27 \ scriptsusing D: \ python27 \ Lib \ Site-packagesfinished processing dependencies for scrapy = 0.14.4t: \ Scrapy-0.14.4>

Verify installation:

 T :\> scrapyscrapy 0.14.4-no active projectusage: scrapy  [Options] [ARGs] available commands: fetch fetch a URL using the scrapy downloader runspider run a self-contained Spider (without creating a project) settings get settings values shell interactive scraping console startproject create new project version print scrapy version view open URL in browser, as seen by scrapyuse "scrapy -h" to see more info about a commandt: \> 
Related Article

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: info-contact@alibabacloud.com and provide relevant evidence. A staff member will contact you within 5 working days.

Tags Index: