A tutorial on using Python's scrapy framework for 10 minutes to climb a beauty map

Brief introduction

Scrapy is a rich, quick and easy to use crawler framework underneath Python. With Scrapy can quickly develop a simple reptile, the official given a simple example is sufficient to prove its strong:

Rapid development

Here's the 10-minute countdown:

1. Initializing the project

Scrapy Startproject MZT CD MZT Scrapy Genspider Meizitu meizitu.com

2. Add Spider Code:

Define Scrapy. Item, add image_urls and images, and prepare for downloading pictures.

Modify Start_urls as the initial page, add parse to process the list page, add Parse_item process project page.

3. Modify the configuration file:

download_delay=1# Add Download Delay configuration item_pipelines={' scrapy.pipelines.images.ImagesPipeline ': 1}# add picture Download pipeline Images_store= '. ' # Set Picture Save directory

4. Run the project:

Scrapy Crawl Meizitu

See, Project Operation effect Chart

Waiting for a while, is the harvest time