Python crawler Primer-scrapy Basic use

Source: Link: http://pan.baidu.com/s/1dEK82hb Password: 9flo
Create Project Scrapy startproject Tutorial Crawl scrapy crawl D Moz Crawl and Save as JSON format Scrapy crawl Dmoz-o items.json-t JSON
scrapy Shell "url/resource" will be able to get response response after loading Response.bodyresponse.headers>>>response.xpath ('//title ') > >>response.xpath ('//title/text () '). Extract ()

Edit Item:

  
 

   
  

  
# -*- coding: utf-8 -*-
   
  

  

   
  

  
# Define here the models for your scraped items
   
  

  
#
   
  

  
# See documentation in:
   
  

  
# http://doc.scrapy.org/en/latest/topics/items.html
   
  

  

   
  

  
import scrapy
   
  

  

   
  

  

   
  

  
class DmozItem(scrapy.Item):
   
  

  
 # define the fields for your item here like:
   
  

  
 # name = scrapy.Field()
   
  

  
 title = scrapy.Field()
   
  

  
 link = scrapy.Field()
   
  

  
 desc = scrapy.Field()
  
 

Edit spider: Crawl and Save as JSON format here, there will be a JSON file in the root directory


Null

Python crawler Primer-scrapy Basic use