Spider is different from crawler.

Intermediary transaction http://www.aliyun.com/zixun/aggregation/6858.html ">seo diagnose Taobao guest cloud host technology Hall

The motivation for writing this story comes from the repetitive robots in the previous article, which reminds me of spider and Crawler (reptiles). The two are the same?

I've read an article before, saying it's not the same, or that it's strictly different. Just now search the Internet, most of the comments said the same. Most of the comments, I do not boil down here, find it online, a lot of it. I will say "This is not the same". Right or wrong, all as a reference, hundred schools of all flowers.

In WebmasterWorld, there has been a post, talking about is spider and crawler. The post begins with a narrative:

Search engines consist of hep Discrete software rs:

SPIDER:A robotic browser like the downloads webpages.

Crawler:a wandering spider that automatically follows links found on pages.

Indexer:a blender like this dissects webpages that are downloaded by spiders.

The Database:a warehouse of the pages downloaded and processed.

Search Engine Results Engine:digs search Results out of the database.

A word summed up its meaning, is: Spider and crawler is not the same.

There is also a point of view in the post, that is, 5 kinds of robots, the name, the role is: Spider, download the page; crawler, follow the inner chain, access to the other end of the link; indexer, download the Web page, datebase, downloaded, processed the Web page of the warehouse; Result engine to find the search results from the database. 5 kinds? I don't know if that's true, but at least it's new to me.

Another speaker said:

Let's talk about how to interpret your page for a bit. If I Follow Brett ' s historical topic, you have three different, types of robots, a spider, Crawler and indexer.

The URI of the Spider comes around and requests. It reads server header information and other on page information. Then The Crawler follows all the links within this domain (those that are found and even). Then the Indexer reads the HTML while making heads and tails of it.

Its speakers believe that there are 3 kinds of robots: spider, Crawler, indexer. The first is spider according to the URI, Access comes in, then reads the header of the server and the head tag of the page. Then, crawler along the inner chain of the Web page found by spider to access the other end of the inner chain. Finally, indexer to read the HTML code.

What do we think about this? I hope this article can play a role in the discussion.