Using code to record the activity of spiders

Summary: Many do seo, often ask why my page is not included, why my snapshot or last week, how Baidu does not update my site. These are just questions to ask. Do SEO people know that the law of the search engine is very

Many do seo, often ask why my page is not included, why my snapshot or last week, how Baidu does not update my site. These are just questions to ask. Do SEO people know that the law of the search engine is very important. You want to know whether the spider has been to your site, when it was, which spider. These are good for your targeted optimization.

Many servers support spider crawling log, but most of the webmaster are virtual machines, some space does not support the recording spider crawling. At this point we need to write code to record the spider's activity patterns.

Find a lot of code on the Internet can not be used correctly, or the function is not perfect.

Just do your own research. Sum up a set of code, hundred percent guaranteed to use.

<?php

function Get_naps_bot ()

{

$useragent = Strtolower ($_server[' http_user_agent '));

if (Strpos ($useragent, ' Googlebot ')!== false) {

Return ' Googlebot ';

}

if (Strpos ($useragent, ' MSNBot ')!== false) {

Return ' MSNbot ';

}

if (Strpos ($useragent, ' slurp ')!== false) {

Return ' Yahoobot ';

}

if (Strpos ($useragent, ' Baiduspider ')!== false) {

Return ' Baiduspider ';

}

if (Strpos ($useragent, ' Sohu-search ')!== false) {

Return ' Sohubot ';

}

if (Strpos ($useragent, ' Lycos ')!== false) {

Return ' Lycos ';

}

if (Strpos ($useragent, ' Robozilla ')!== false) {

Return ' Robozilla ';

}

return false;

}

function Nowtime () {

$date =date ("y-m-d.g:i:s");

return $date;

}

$searchbot = Get_naps_bot ();

if ($searchbot) {

$TLC _thispage = addslashes ($_server[' http_user_agent '));

$url =$_server[' Http_referer '];

$file = "Robotslogs.txt";

$time =nowtime ();

$data =fopen ($file, "a");

$PR = "$_server[request_uri]";

Fwrite ($data, "Time: $time Robot: $searchbot URL: $TLC _thispage\n page: $PR \ r \ n");

Fclose ($data);

}

?>

The above code than the online version of a more than a record search engine request page records, you can see the spider every time to crawl is that page. More conducive to the analysis of spiders on the weight of the site and the different columns and pages of the favor.

SEO is ultimately from the actual start, with the data to speak, rather than imagining. All optimization is based on data, and all the purpose of optimization is to get data.

This article is reproduced by CEO Vane, please specify from http://www.ceologo.com/seo/371.html