Install and build the Scrapy 6.4 environment in CentOS 0.22

Install and build the Scrapy 6.4 environment in CentOS 0.22

 

Scrapy is an open-source python standalone crawler with the twisted framework. This crawler actually contains a toolkit for most web crawlers to download and extract.

The installation and setup of the Scrapy 6.4 environment in CentOS 0.22 is recorded, and we hope to help you.

 

1. Install Python2.7.6

1. Update the CentOS lib Library File

Yum-y update

2. Install the Development Kit

Yum groupinstall-y development

3. Install the extension package

Yum install-y zlib-dev openssl-devel sqlite-devel bzip2-devel

4. Download Python2.7.6

Wget http://www.python.org/ftp/python/2.7.6/Python-2.7.6.tar.xz

5. Decompress Python2.7.6

Tar-xvf Python-2.7.6.tar.xz

6. Go to the Python2.7.6 directory.

Cd Python-2.7.6

7. Compile and install Python2.7.6

./Configure

Make & make altinstall

Ii. Solve the problem that Yum does not support Python2.7.

1. Back up the old Python Program

Mv/usr/bin/python/usr/bin/python2.6.6

2. Create a New Python Link

Ln-s/usr/local/bin/python2.7/usr/bin/python

3. Modify the/usr/bin/yum File

Vi/usr/bin/yum

Set the #! Of the first line #! /Usr/bin/python #! /Usr/bin/python2.6.6

3. Install setuptools and pip

1. Download setuptools

Wget -- no-check-certificate https://pypi.python.org/packages/source/s/setuptools/setuptools-1.4.2.tar.gz

2. Decompress setuptools

Tar-xvf setuptools-1.4.2.tar.gz

3. Go to the setuptools directory.

Cd setuptools-1.4.2

4. Install setuptools through the installed Python2.7.6

Python setup. py install

5. Install pip

Easy_install pip

6. curl https://raw.github.com/pypa/pip/master/contrib/get-pip.py | python2.7-

4. Install libxslt-devel to support lxml

Yum install libxslt-devel

5. Install Scrapy

Pip install scrapy

This article permanently updates the link address: