You can clone all the source code on GitHub.
Github:https://github.com/williamzxl/scrapy_crawlmeizitu
Scrapy Official Document: http://scrapy-chs.readthedocs.io/zh_CN/latest/index.html
Basically, follow the documentation process to go through the basic will be used.
STEP1:
Before you begin a crawl, you must create a new Scrapy project. Enter the directory where you want to store the code, and run the following command:
Startproject Crawlmeizitu
The command creates a directory that contains the following content tutorial
:
crawlmeizitu/scrapy. CFG crawlmeizitu / __init__. PY items. PY pipelines. PY settings. PY
middlewares.py spiders/__init__.py CD crawlmeizitu
/span>
Meizituhttp://www.meizitu.com/a/list_1_1.html
The command creates a directory that contains the following content tutorial
:
crawlmeizitu/scrapy. CFG crawlmeizitu/
__init__< Span class= "O". py items. PY pipelines. PY settings. PY
middlewares.py spiders/
meizitu.py __init__.py
Our main editors are as shown in the arrows:
Main.py was later added, adding two orders, mainly for the convenience of operation.
STEP2: Edit settings as shown in
STEP3: Edit Items.
STEP4: Edit Pipelines
STEP5: Edit the main program of the Meizitu.
Python uses scrapy crawler frame to crawl pictures and save local (sister map)