Python+scrapy Crawl Amazon Mobile Products

1 #-*-coding:utf-8-*-2 3 #Define Here the models for your scraped items4 #5 #See documentation in:6 #http://doc.scrapy.org/en/latest/topics/items.html7 8 Importscrapy9 Ten  One classAmazonitem (scrapy. Item): A     #Define the fields for your item here is like: -     #name = Scrapy. Field () -description=Scrapy. Field () thePrice=Scrapy. Field () -Url=Scrapy. Field () -Value=scrapy. Field ()

1 #!/usr/bin/python2 3 Importscrapy4 classAmazonspider (scrapy. Spider):5Name='Amazon'6allowd_domains=['amazon.cn']7start_urls=['http://www.amazon.cn/s/ref=sv_cps_0?ie=UTF8&node=665002051&page=1']8     defParse (self,response):9         Try:TenPage=response.xpath ("//span[@class = ' pagndisabled ']/text ()"). Extract () [0] One         except: A             Pass -          forIteminchResponse.xpath ("//li[@class = ' S-result-item ')"): -Title=item.xpath ("./div/div[2]/div/a/h2/text ()"). Extract () [0] thePrice=item.xpath ("./div/div[3]/div[1]/a/span[1]/text ()"). Extract () [0] -Url=item.xpath ("./div/div[1]/div/div/a[1]/@href"). Extract () [0] -             Printtitle -             Print Price +             PrintURL -          forIinchRange (int (page)): +Request=scrapy. Request ('http://www.amazon.cn/s/ref=sv_cps_0?ie=UTF8&node=665002051&page='+str (i), callback=self.parse) A             yieldRequest at             

Python+scrapy Crawl Amazon Mobile Products