RESTful Web apis_2013
The crawler simulates a very curious but not very picky human. Give it a URL to Startwith, and it'll fetch a representation. Then it'll follow all the links it can find to getmore representations. It would do this recursively, until there is no more representations to be had. The Mapmaker client from earlier on this chapter are a kind of crawler for maze+xml documents. The spiders used by search engines is crawlers for HTML documents. It's quite difficult to write a crawler for an API that doesn ' t use hypermedia. But the can write a crawler for a hypermedia-based API without even understanding that API ' s linkrelations. Generally speaking, a crawler would only trigger the state transitions that is safe. Otherwise,there ' s no telling what will happen to resource state. a crawler that sent a DELETE request to every resource it encountered, just to see what Happ Ened, would be a terrible client.
A crawler that sent A deleterequest to every resource it encountered