Scrapy Series Tutorial One--scrapy introduction and scrapy Installation

1. What can scrapy do?

Scrapy is an application framework written to crawl Web site data and extract structural data. Can be applied in a series of programs including data mining, information processing, or storing historical data. It was originally designed for page fetching (more specifically, network crawling) and could also be applied to get the data returned by the API (for example, Amazon Associates Web Services) or a generic web crawler.

2. Scrapy Related Documents

Scrapy Chinese Documents

http://scrapy-chs.readthedocs.org/zh_CN/0.24/

Scrapy Official Website documents

http://doc.scrapy.org/

2. Scrapy installation

The following installation steps assume that you have installed the following programs:

• Python 2.7
• Python Package:pip
• lxml. Most Linux distributions bring their own lxml. If missing, see http://lxml.de/installation.html
• Openssl. Systems other than Windows (see the Platform Installation Guide) are already available.

You can use Pip to install Scrapy (it is recommended to use PIP to install the Python package).

Pip Install Scrapy

Reference Tutorials

Http://jingyan.baidu.com/article/f3ad7d0f129c7809c2345b56.html

Specific steps:

1. Open the terminal, execute the following command python, next is import lxml, import OpenSSL. Such as. If the import does not have an error, the system is already self-brought.

2. In order to ensure the following installation can be successful. We do this first: sudo apt-get install Python-dev and sudo apt-get install Libevent-dev.

3. The next step is to install PIP, execute: Apt-get install PYTHON-PIP.

4. Install the scrapy. Execute: Pip install Scrapy,

5. Verify that it is successful

Terminal Execution scrapy Command

Scrapy Series Tutorial One--scrapy introduction and scrapy Installation