Python uses scrapy crawler frame to crawl pictures and save local (sister map)

You can clone all the source code on GitHub.

Github:https://github.com/williamzxl/scrapy_crawlmeizitu

Scrapy Official Document: http://scrapy-chs.readthedocs.io/zh_CN/latest/index.html

Basically, follow the documentation process to go through the basic will be used.

STEP1:

Before you begin a crawl, you must create a new Scrapy project. Enter the directory where you want to store the code, and run the following command:

Startproject Crawlmeizitu

The command creates a directory that contains the following content tutorial :

 crawlmeizitu/scrapy. CFG crawlmeizitu   / __init__. PY items. PY pipelines. PY settings. PY 
 middlewares.py spiders/__init__.py  CD  crawlmeizitu         
                 /span>         

Meizituhttp://www.meizitu.com/a/list_1_1.html

The command creates a directory that contains the following content tutorial :

crawlmeizitu/scrapy. CFG crawlmeizitu/
 __init__< Span class= "O". py items. PY pipelines. PY settings. PY 
 middlewares.py spiders/
 meizitu.py          __init__.py             

Our main editors are as shown in the arrows:


Main.py was later added, adding two orders, mainly for the convenience of operation.

STEP2: Edit settings as shown in


STEP3: Edit Items.


STEP4: Edit Pipelines


STEP5: Edit the main program of the Meizitu.

Python uses scrapy crawler frame to crawl pictures and save local (sister map)