Using code to record the activity of spiders

Source: Internet
Author: User
Keywords SEO why very
Tags activity code data date function not included questions return

Summary: Many do seo, often ask why my page is not included, why my snapshot or last week, how Baidu does not update my site. These are just questions to ask. Do SEO people know that the law of the search engine is very

Many do seo, often ask why my page is not included, why my snapshot or last week, how Baidu does not update my site. These are just questions to ask. Do SEO people know that the law of the search engine is very important. You want to know whether the spider has been to your site, when it was, which spider. These are good for your targeted optimization.

Many servers support spider crawling log, but most of the webmaster are virtual machines, some space does not support the recording spider crawling. At this point we need to write code to record the spider's activity patterns.

Find a lot of code on the Internet can not be used correctly, or the function is not perfect.

Just do your own research. Sum up a set of code, hundred percent guaranteed to use.

<?php

function Get_naps_bot ()

{

$useragent = Strtolower ($_server[' http_user_agent '));

if (Strpos ($useragent, ' Googlebot ')!== false) {

Return ' Googlebot ';

}

if (Strpos ($useragent, ' MSNBot ')!== false) {

Return ' MSNbot ';

}

if (Strpos ($useragent, ' slurp ')!== false) {

Return ' Yahoobot ';

}

if (Strpos ($useragent, ' Baiduspider ')!== false) {

Return ' Baiduspider ';

}

if (Strpos ($useragent, ' Sohu-search ')!== false) {

Return ' Sohubot ';

}

if (Strpos ($useragent, ' Lycos ')!== false) {

Return ' Lycos ';

}

if (Strpos ($useragent, ' Robozilla ')!== false) {

Return ' Robozilla ';

}

return false;

}

function Nowtime () {

$date =date ("y-m-d.g:i:s");

return $date;

}

$searchbot = Get_naps_bot ();

if ($searchbot) {

$TLC _thispage = addslashes ($_server[' http_user_agent '));

$url =$_server[' Http_referer '];

$file = "Robotslogs.txt";

$time =nowtime ();

$data =fopen ($file, "a");

$PR = "$_server[request_uri]";

Fwrite ($data, "Time: $time Robot: $searchbot URL: $TLC _thispage\n page: $PR \ r \ n");

Fclose ($data);

}

?>

The above code than the online version of a more than a record search engine request page records, you can see the spider every time to crawl is that page. More conducive to the analysis of spiders on the weight of the site and the different columns and pages of the favor.

SEO is ultimately from the actual start, with the data to speak, rather than imagining. All optimization is based on data, and all the purpose of optimization is to get data.

This article is reproduced by CEO Vane, please specify from http://www.ceologo.com/seo/371.html

Related Article

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: info-contact@alibabacloud.com and provide relevant evidence. A staff member will contact you within 5 working days.

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.