Scrapy installation--------Windows, Linux, Mac and other operating platforms

Scrapy Installation

There are several ways to install scrapy, which supports the Python2.7 version and above or the Python3.3 version and above. Below the PY3 environment, the scrapy installation process.

Scrapy relies on more cubby, at least to rely on the library Twisted 14.0,lxml 3.4,pyopenssl 0.14. Different platform environments are not the same, so be sure to install some basic libraries, especially windows, before installing.

First, Anaconda

This approach is a relatively simple way to install scrapy (especially for Windows), which you can install using this method. You can also choose the installation method for the dedicated platform below.

　　Anaconda is a Python release that contains a common data science library , and if it is not installed, it can be installed on the website https://www.continuum.io/downloads download the corresponding platform package.

If it is already installed, you can install scrapy with the Conda command.

Install the following:

First open the Anaconda Anaconda Prompt input

Conda Install Scrapy

Indicates successful installation:

　　

Second, Windows1. Installation lxml

The best way to install is through the wheel file to install, http://www.lfd.uci.edu/~gohlke/pythonlibs/, this site is really the Gospel of Windows users, basically the Python library has inside, called him as Python Universal Library website . Find lxml related files (ctrl+f) from this website, if python3.5 version, Windows 64-bit system, then find lxml? 3.7.2? Cp35? cp35m? WIN_AMD64.WHL This file to download, install via PIP.

　　

After downloading, run the following command to install:

PIP3 Install wheelpip3 install lxml? 3.7.2? CP35?CP35M?WIN_AMD64.WHL# If it's a python2 version, PIP3 changes to PIP2

You can complete the installation of lxml, and replace the file name with other files.

2. Installing Pyopenssl

Official website Download wheel file, https://pypi.python.org/pypi/pyOpenSSL#downloads, download. whl file, install it.

PIP3 Install PYOPENSSL-16.2.0-PY2.PY3-NONE-ANY.WHL

3. Installing twisted

In the same vein, download  http://www.lfd.uci.edu/~gohlke/pythonlibs/#twisted   Install from the Python Universal website. If it is python3.5 version, Windows 64-bit system, then find  twisted?17.1.0? Cp35? Cp35m? WIN_AMD64.WHL
  This file is downloaded via PIP installation.

PIP3 install Twisted?  17.1.0?  Cp35?  cp35m?

4. Installing Pywin32

Download the corresponding version of the installation package from the official website https://sourceforge.net/projects/pywin32/files/pywin32/Build%20220/.

5. Installing Scrapy

Finally install Scrapy, still use PIP, command as follows:

PIP3 Install Scrapy

Third, CentOS, RedHat, Fedora

Depending on the library installation, make sure that some of the required class libraries are installed and run the following command:

sudo yum groupinstall development toolssudo yum install python34-devel epel-release libxslt-devel libxml2-devel Openssldevel
PIP3 Install Scrapy

Iv. Ubuntu, Debian, Deepin

Depending on the library installation, make sure that some of the required class libraries are installed and run the following command:

sudo apt-get install build-essential python3-dev libssl-dev libffi-

PIP3 Install Scrapy

Five, Mac OS

Relying on library installation, build scrapy dependent libraries on Mac requires C compiler and development header file, he is generally provided by Xcode, run the following command to install:

Xcode-select--install
PIP3 Install Scrapy

Verify:

After installation, at the command line input scrapy, if a similar result appears, then long congratulations on your installation success scrapy.

　　

Vi. Common Mistakes

• Pkg_resources. Versionconflict: (six 1.5.2 (/usr/lib/python3/dist-packages), Requirement.parse (' six>=1.6.0 '))
  Six package version is too low, six package is a compatible with Python2 and 3 of the library, upgrade six package.
  

  sudo pip3 install-u Six

• C/_cffi_backend.c:15:17:fatal error:ffi.h:no such file or directory
  Missing Libffi This library, what is Libffi? The full name of the "FFI" is foreign Function Interface, which usually refers to code that allows code written in one language to call another language. The Linffi library only provides the lowest-level, architecture-related, complete "FFI".
  Installation:
  Ubuntu, Debian: sudo apt-get install build-essential libssl-dev libffi-dev Python3-dev
  CentOS, RedHat:sudo yum install gcc libffi-devel python-devel openssl-devel
  
• Importerror:no module named ' Cryptography '
  This is an associated component that lacks encryption and can be installed with PIP.
  

  sudo pip3 Install cryptography

• Importerror:no module named ' Packaging
  This package is missing packaging, which provides the core functionality of the Python package. You can install with PIP.
  

  sudo pip3 install packaging

• Importerror:no module named ' Appdirs '
  Missing Appdirs This package, which is used to determine the file directory, can be installed with PIP alone.
  

  sudo pip3 install Appdirs

Today's filial piety
Source: Http://www.cnblogs.com/jinxiao-pu
This article is copyrighted by the author and the blog Park, Welcome to reprint, but without the consent of the author must retain this paragraph, and in the article page obvious location to the original link.

Scrapy installation--------Windows, Linux, Mac and other operating platforms