Intermediary transaction SEO diagnosis Taobao guest Cloud host technology Hall
search engine robot name
Googlebot
Google
Baiduspider
Baiduspider
msnbot or MSNBot msnbot
ISAAC Ding
P.arthur
yahoo!+slurp
Sohu-search
spiderman = (yahoo)
Gaisbot
Psbot
Msiecrawler
Sbl-bot
Mediapartners-google
Becomebot
8fang.net
Gigabot
spider.net
Wget
Haoransoft
Swoogle
UltraSEEK
Java
Linkwalker
Openfind or Openbot
Aibot
Nutch
Lycos
Robozilla
crawler
Web Indexing Robot
Surveybot 2.3 (Whois Source)
Fast-webcrawler
Scooter
slurp
Alexa
Wisenutbot
Ibm_planetwide
Turn It in
Larbin
Jeeves
Crawl
Voila
Appie
Google AdSense
Spider
Robot
Bbot
Newzcrawler
=============================
the following unauthenticated
--------------------------------------------------------------------------------
abachobot=abacho.com
abcdatos_botlink=abcdatos.com
http://www.abcdatos.com/botlink/=Abcdatos.com
aesop_com_spiderman=aesop.com
ah-ha.com Crawler (crawler@ah-ha.com) =ah-ha.com
ia_archiver=archive.org
scooter=altavista.com
mercator=altavista.com
scooter2_mercator_3-1.0=altavista.com
Roach.smo.av.com-1.0=altavista.com
tv_merc_resh_26_1_d-1.0=altavista.com
altavista-intranet=altavista.co.uk
jan.gelin@av.com=altavista.co.uk
fast-webcrawler=alltheweb.com
crawler@fast.no=alltheweb.com
Acoon robot=acoon.de
antibot=antisearch.net
atomz=atomz.com
Buscaplus robi=buscaplus.com
canseek/=canseek.ca
support@canseek.ca=canseek.ca
christcrawler=christcrawler.com
crawler=crawler.de
admin@crawler.de=crawler.de
daadle.com robot/=daadle.com
rabot=daum.net
agent-admin/=daum.net
phortse@hanmail.net=daum.net
contact/jylee@kies.co.kr=kies.co.kr
deepindex=deepindex.com
dittospyder=ditto.com
jack=domanova.co.uk
Speedy spider=entireweb.com
architextspider=excite.com
architectspider=excite.com
arachnoidea=euroseek.net
arachnoidea@euroseek.net=euroseek.net
ezresult=ezresults.com
Fast partnersite crawler=fastsearch.net
FAST Data Search crawler=fastsearch.net
kit-fireball=fireball.de
fybersearch=fybersearch.com
galaxybot=galaxy.com
geckobot=geckobot.com
gencrawler=gendoor.com
geonabot=geona.com
googlebot=google.com
googlebot@googlebot.com=google.com
google=google.com
moget/2.0=goo.ne.jp
moget@goo.ne.jp=goo.ne.jp
aranha=girafa.com
Slurp.so/1.0=yahoo
Slurp@inktomi.com=yahoo
Slurp/2.0j=yahoo
Www.inktomisearch.com=Yahoo
Slurp/2.0-kitehourly=yahoo
Slurp/2.0-owlweekly=yahoo
Spider@aeneid.com=yahoo
Slurp/3.0-au=yahoo
Toutatis 2.5-2=hoppa.com
hubater=hubat.com
iltrovatore-setaccio=iltrovatore.it
incywincy=incywincy.com
ultraseek=infoseek.com
InfoSeek sidewinder=infoseek.com
mole2/1.0=intags.de
webmaster@intags.de=intags.de
mp3bot=mp3bot.de
c-pbwf-ip3000.com-crawler=ip3000.com
ip3000.com-crawler=ip3000.com
kuloko-bot/0.2=kuloko.com
lnspiderguy=lexis-nexis.com
netresearchserver=look.com
mantraagent=looksmart.com
netresearchserver=loopimprovements.com
Lycos_spider_ (Rex) =lycos.com
joocerbot=joocer.com
henrythemiragorobot=mirago.co.uk
Mozdex/=mozdex.com
msnbot/0.1=msn
gulliver=northernlight.com
objectssearch/0.01=objectssearch.com
picosearch/=picosearch.com
pjspider=portaljuice.com
diibot=powerinter.net
nttdirectory_robot=navi.ocn.ne.jp
super-robot@super.navi.ocn.ne.jp=navi.ocn.ne.jp
griffon=super.navi.ocn.ne.jp
griffon@super.navi.ocn.ne.jp=super.navi.ocn.ne.jp
spider/maxbot.com=maxbot.com
admin@maxbot.com=maxbot.com
Gazz/1.0=unknown Spider
Gazz@nttrd.com=unknown Spider
nationaldirectory-superspider=nationaldirectory.com
Dloader (Naverrobot)/=naver.com
Dumrobo (Naverrobot)/=naver.com
Openfind piranha=openfind.com
shark=openfind.com
robot-response@openfind.com.tw=openfind.com.tw
openbot/=openfind.com.tw
psbot=picsearch.org
crawlerboy=pinpoint.com
ip3000.com=petersnews.com
Alkalinebot=alkalinebot
Fluffy the spider=searchhippo.com
info@searchhippo.com=searchhippo.com
scrubby/=scrubtheweb.com
asterias=singingfish.com
speedfind Rambot xtreme=speedfind.de
kototoi/0.1=s.u-tokyo.ac.jp
searchspider/=searchspider.com
sightquestbot/=sightquest.com
spider_monkey/=spidermonkey.ca
Surfnomore Spider v1.1=surfnomore.com
robot@supersnooper.com=supersnooper.com
teoma_agent1=teoma.com
teoma_admin@hawkholdings.com=teoma.com
teradex_mapper=mapper.teradex.com
mapper@teradex.com=mapper.teradex.com
esismartspider=travel-finder.com
Spider traficdublu=traficdublu.ro
Tutorial crawler=tutorgig.com
UK Searcher spider=uksearcher.co.uk
Vivante Link checker=vivante.com
appie=walhello.com
nazilla=websmostlinked.com
www.WebWombat.com.au=webwombat.com.au
marvin/infoseek=webseek.de
marvin-team@webseek.de=webseek.de
muscatferret=webtop.com
whizbang! Lab=whizbanglabs.com
zyborg=wisenut.com
WIRE webrefiner=wire.co.uk
wscbot=worldsearchcenter.com
yandex=yandex.com
yellopet-spider=yellowpet.com
iron33=verno.ueda.info.waseda.ac.jp/
Alink=link Checkers
Ameta=link Checker
aspsearch URL checker=link Checker
Blogbot=link Checker
Bmchecker=link Checker
Bookmark Buddy=link Checker
Check&get=link Checker
Checkweb=link Checker
Cnet_snoop=link Checker
CSE HTML validator=link Checker
Drkspider=link Checker
DISCo Watchman=link Checker
Doctorhtml=link Checker
Email extractor=email Extractor
emailsiphon=email Extractor
emailwolf=email Extractor
Favorg=link Checker
Favorites Sweeper=link Checker
Freshlinks.exe=link Checker
Funnel Web profiler=link Checker
Html Link validator=link Checker
the Informant=link Checker
the Intraformant=link Checker
Internetlinkagent=link Checker
Internetperiscope=link Checker
Javelink=link Checker
Jdwhatsnew.cgi=link Checker
jrts Check Favorites utility=link Checker
Lambda Linkcheck=link Checker
Linklint-checkonly=link Checker
Linkalarm=link Checker
Linkbot=link Checker
Linkman=link Checker
Linkprover=link Checker
Links=link Checker
Linkscan Server=link Checker
Linksweeper=link Checker
Link Valet Online=link Checker
linkverify Spider=link Checker
Linkwalker=link Checker
Morning Paper=link Checker
Moveannouncer=link Checker
Netlookout=link Checker
Netmechanic=link Checker
Www.elsop.com=Link Checker
Netmind-minder=link Checker
Netmonitor=link Checker
netprospector Javacrawler=link Checker
Online link Validator=link Checker
Rational Sitecheck=link Checker
Robozilla=link Checker
Rpt-httpclient=link Checker
Surfmaster=link Checker
Syncit=link Checker
Watchfire Webxm=link Checker
watznew Agent=link Checker
Website-watcher=link Checker
WebTrends Link analyzer=link Checker
weblink Scanner=link Checker
Xenu ' s Link sleuth=link Checker
W3c_validator=link Validator
Wdg_validator/=link Validator
Tooter=link Validator
citenikbot/=citenik.co.uk
clips-index=clips-index.imag.fr/
Computer_and_automation_research_institute_crawler=research Bot
cosmos=xyleme.com
robot@xyleme.com=xyleme.com
Diagem/=diagem
Digimarc webreader=digimarc.com
echo!/2.0=voila.com
finalerobot=expressus.com
robot-master@expressus.com=expressus.com
Ideare-signsite=ideare.com
gentlespider=research.att.com
gulper Web bot=gulper web Bot
Larbin=unknown Spider
sebastien.ailleret@inria.fr=inria.fr
Ghi@lcs.mit.edu=unknown Spider
Multitext=multitext
NEC Agent=nec Agent
Ontospider=ontospider
sherlock_spider=sherlock.com.cn
Steeler=steeler
ru-robot=rutgers.edu
0.1_hseo (at) cs.rutgers.edu=rutgers.edu
Webgather=webgather
Xyro=xyro
Xcrawler@inria.fr=unknown Spider
Zao/0.2=zao
Adsarobot=adsarobot
Answerchase=answerchase
Aspseek=aspseek
Avsearch=avsearch
Checkbot=checkbot
Daviesbot=daviesbot
deepweb=deepweb.com
gigabaz=brainbot.com
Gigabazvstheweb=brainbot.com
crawler@brainbot.com=brainbot.com
giskard=oralco.com
Internetseer=internetseer
Ipiumbot=ipiumbot
Insumascout=insumascout
Katriona=katriona
Leia=leia
lexibot=lexibot.com
Metabot=metabot
Netcruiser=netcruiser
npbot=nameprotect.com
Netzippy=netzippy
nzbot=navigationzone.com
opencola=opencola.com
Oxxbot1=oxxbot
Pansophica=pansophica
Phoaks=phoaks
Picgrabber=picgrabber
pictureofinternet=pictureofinternet
Erik@malfunction.org=unknown Spider
Pintaspider=pintaspider
Polybot=polybot
Squid=squid
Sqworm=sqworm
Tawwwantula=tawwwantula
Teracrawl=teracrawl
turnitinbot=turnitin.com
ucmore=ucmore.com
Udmsearch=mnogosearch
unlostbot=unlost.com
urlblaze=urlblaze.net
Urlscope=urlscope
Vagabondo=vagabondo
Vspider=vspider
Wavetools=wavetools
Webbandit=webbandit
webclipping.com=webclipping.com
Webcollage=webcollage
Webcompass=webcompass
Webgenie=webgenie
Web Magnet=unknown Spider
Webminer=unknown Spider
Webpush=unknown Spider
Websymmetrix=unknown Spider
Webrank=unknown Spider
Webwasher=unknown Spider
Whostalking=unknown Spider
anzwerscrawl/2.0=anzwers
fido/1.0 harvest/1.4.pl2=planet Search
Gais robot/1.0b2=seednet
googlebot/1.0=google.com
Gulliver/1.2=northern Light
Infoseek Sidewinder/0.9=infoseek
Kit_fireball/2.0=fireball
Lwp-trivial/1.27=search 4 Free
Lycos_spider_ (Rex)/3.0=lycos
Scooter/1.0=altavista
scooter/1.0 Scooter@pa.dec.com=altavista
scooter/1.1 (custom) =altavista
scooter/2.0 g.r.a.b X2.0=altavista
scooter/2.0 g.r.a.b V1.1.0=altavista
search.at v1.2=search.at
Inktomi=inktomi Spider
Swisssearch V1.2=swisssearch
the Informant=the informant
Ultraseek=infoseek
webcrawler/3.0 Robot Libwww/5.0a=webcrawler
Webcrawler-addurl/2.0=webcrawler
Wisewire=wisewire
Wisewire-alpha-1.0=wisewire
Wisewire-alpha-spider=wisewire
Wisewire-alpha12-spider971219a=wisewire
Wisewire-alpha12-spider (971223a) =wisewire
Wisewire-hotspider-1.0=wisewire
Wisewire-spider=wisewire
Wisewire-spider-1.0=wisewire
Wisewire-spider2=wisewire
Wisewire-widow-1.0=wisewire
Wisewire-widow-1.0r=wisewire
Wisewire-widow-1.0-alpha12=wisewire
cherrypickerse/1.0=email Extractor
cherrypickerelite/1.0=email Extractor
Crescent Internet ToolPak HTTP OLE control v.1.0=email Extractor
emailcollector/1.0=email Extractor
Emailwolf 1.00=email Extractor
extractorpro=email Extractor
Ask Jeeves=ask Jeeves
lycos=lycos.com
Whatuseek=what You
Wisenutbot=looksmart
msnbot=msn
Gigablast=gigablast
Gigabot=gigablast
archive_org=archive.org
Jeeves=ask Jeeves
asterias=singingfish Spider
Slurp=inktomi Spider
Zyborg=looksmart Bot
Baiduspider=baidu
