[Email protected]:/prj/pyscrapy/a$ scrapy startproject Helloword
New scrapy project ' Helloword ' created in:
/cygdrive/e/01.prj/pyscrapy/a/helloword
You can start your first spider with:
CD Helloword
Scrapy Genspider Example example.com
[Email protected]:/prj/pyscrapy/a/helloword$ scrapy genspider Baidu www.baidu.com
Created Spider ' Baidu ' using template ' Basic ' in module:
Helloword.spiders.baidu
Problem:
[Email protected]:/prj/pyscrapy/a/tutorial$ scrapy Crawl DMOZ
/cygdrive/e/01.prj/pyscrapy/a/tutorial/tutorial/spiders/dmoz_spider.py:3: scrapydeprecationwarning: Tutorial.spiders.dmoz_spider. Dmozspider inherits from deprecated class Scrapy.spider.BaseSpider, please inherit from Scrapy.spider.Spider. (Warning subclass, there may others)
Class Dmozspider (Basespider):
2014-12-17 11:32:38+0000 [scrapy] info:scrapy 0.24.4 started (bot:tutorial)
2014-12-17 11:32:38+0000 [Scrapy] info:optional features Available:ssl, HTTP11
2014-12-17 11:32:38+0000 [scrapy] Info:overridden settings: {' newspider_module ': ' tutorial.spiders ', ' SPIDER_MODULES ' : [' tutorial.spiders '], ' bot_name ': ' Tutorial '}
2014-12-17 11:32:40+0000 [scrapy] info:enabled extensions:logstats, Telnetconsole, Closespider, WebService, CoreStats, Spiderstate
2014-12-17 11:32:41+0000 [scrapy] info:enabled downloader middlewares:httpauthmiddleware, Downloadtimeoutmiddleware, Useragentmiddleware, Retrymiddleware, Defaultheadersmiddleware, Metarefreshmiddleware, HttpCompressionMiddleware, Redirectmiddleware, Cookiesmiddleware, Chunkedtransfermiddleware, Downloaderstats
2014-12-17 11:32:41+0000 [scrapy] info:enabled spider middlewares:httperrormiddleware, Offsitemiddleware, Referermiddleware, Urllengthmiddleware, Depthmiddleware
2014-12-17 11:32:41+0000 [scrapy] info:enabled Item Pipelines:
2014-12-17 11:32:41+0000 [DMOZ] Info:spider opened
2014-12-17 11:32:41+0000 [DMOZ] info:crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
2014-12-17 11:32:41+0000 [scrapy] debug:telnet console listening on 127.0.0.1:6023
2014-12-17 11:32:41+0000 [scrapy] debug:web service listening on 127.0.0.1:6080
Iconv [Options ...] [File ...]
The following options are available:
Input/output format specification:
-F,--from-code= name original text encoding
-T,--to-code= name output encoding
Information:
-L,--list enumeration of all known character sets
Output control:
-C ignores invalid characters from output
-O,--output=file output file
-S,--silent off warning
--verbose Printing Progress Information
Iconv-f utf-8-T gb2312/server_test/reports/software_.txt >/server_test/reports/software_asserts.txt
First Scrapy PRJ