How to install scrapy in Ubuntu and Windows

How to install scrapy in Ubuntu and Windows

Scrapy is an open source package in python. You must install it first. First of all, install the python environment .. Ubuntu comes with a python environment by default. You need to install and configure environment variables in Windows.

1. Windows

Scrapy requires many dependent packages in Windows.

(1) pywin32

Pywin32: https://sourceforge.net/projects/pywin32/files/pywin32/

(2) twisted

Of course, the asynchronous communication framework of twisted also requires some dependent packages (= ).

Zope. interface and pyopenssl

Zope: https://pypi.python.org/pypi/zope.interface#download

Pyopenssl: https://pypi.python.org/pypi/pyOpenSSL

Twisted: http://twistedmatrix.com/trac/wiki/Downloads

(3) lxml

: Https://pypi.python.org/pypi/lxml/

Ps: if the extension name of .whlis reached, you only need to change it to .rar, decompress the package, and place the folder in the python system environment.

(4) install scrapy

There are many ways to install scrapy. You can use easytool.

I installed it with pip: pip install scrapy

2. Ubuntu

(1). Upgrade python to 2.7.11 (with pip) --- optional or not

Wget https://www.python.org/ftp/python/2.7.11/Python-2.7.11.tgz

Tar-xvf Python-2.7.11.tgz

Cd Python-2.7.11

./Configure & make install & make clean #

(2) install the dependency package

Apt-get install python-dev libxml2-dev libxslt1-dev libssl-dev libffi-dev #

(3) install scrapy

Pip install Scrapy