1 #-*-coding:utf-8-*-2 3 #Define Here the models for your scraped items4 #5 #See documentation in:6 #http://doc.scrapy.org/en/latest/topics/items.html7 8 Importscrapy9 Ten One classAmazonitem (scrapy. Item): A #Define the fields for your item here is like: - #name = Scrapy. Field () -description=Scrapy. Field () thePrice=Scrapy. Field () -Url=Scrapy. Field () -Value=scrapy. Field ()
1 #!/usr/bin/python2 3 Importscrapy4 classAmazonspider (scrapy. Spider):5Name='Amazon'6allowd_domains=['amazon.cn']7start_urls=['http://www.amazon.cn/s/ref=sv_cps_0?ie=UTF8&node=665002051&page=1']8 defParse (self,response):9 Try:TenPage=response.xpath ("//span[@class = ' pagndisabled ']/text ()"). Extract () [0] One except: A Pass - forIteminchResponse.xpath ("//li[@class = ' S-result-item ')"): -Title=item.xpath ("./div/div[2]/div/a/h2/text ()"). Extract () [0] thePrice=item.xpath ("./div/div[3]/div[1]/a/span[1]/text ()"). Extract () [0] -Url=item.xpath ("./div/div[1]/div/div/a[1]/@href"). Extract () [0] - Printtitle - Print Price + PrintURL - forIinchRange (int (page)): +Request=scrapy. Request ('http://www.amazon.cn/s/ref=sv_cps_0?ie=UTF8&node=665002051&page='+str (i), callback=self.parse) A yieldRequest at
Python+scrapy Crawl Amazon Mobile Products