Scrapy, scrapy tutorial

Scrapy, scrapy tutorial

 

Create a project

GenerallyScrapyThe first thing about the tool is to create your Scrapy project:

Scrapy startproject myproject

This command willMyprojectCreate a Scrapy project in the directory.

Next, go to the project directory:

Cd myproject

In this case, you can useScrapyCommand to manage and control your project.

Control Project

You can useScrapyTools to control and manage them.

For example, create a new spider:

Scrapy genspider mydomain mydomain.com

Crawl

• Syntax:Scrapy crawl <spider>
• Project required or not:Yes

Use spider for crawling.

Example:

$ Scrapy crawl myspider [... myspider starts crawler...]