Php code sharing for crawling spider traces

Php code sharing for crawling spider traces

This article describes how to use php to capture Spider traces. For more information, see. Use php code to analyze spider crawlers in web logs. the code is as follows: 'Googlebot ', 'baidu' => 'baidider Ider', 'Yahoo '=> 'Yahoo slurp', 'soso' => 'sosospider ', 'MSN '=> 'msnbot', 'altavista' => 'Scooter', 'sogou' => 'sogou spider ', 'yodao' => 'yodaobot '); $ userAgent = strtolower ($ _ SERVER ['http _ USER_AGENT ']); foreach ($ bots as $ k = >$ v) {if (strstr ($ v, $ userAgent) {return $ k; break;} return false;} // by bbs.it-home.org//the species of spider crawlers are kept alive. // Prevent collection based on whether HTTP_USER_AGENT is empty during Collection // capture Spider crawlers -- by bbs.it-home.org $ spi = isSpider (); if ($ spi) {$ tlc_thispage = addslashes ($ _ SERVER ['http _ USER_AGENT ']); $ file = 'robot.txt'; $ time = date ('Y-m-d H: I: s ', mktime (); $ handle = fopen ($ file, 'A +'); $ PR = $ _ SERVER ['request _ URI ']; fwrite ($ handle, "Time: {$ time} ROBOT: {$ spi} AGENT: {$ tlc_thispage} URL: {$ PR} \ n \ r "); fclose ($ handle) ;}?>