File_get_contents solution for capturing webpage garbled code _ PHP Tutorial-php Tutorial

Source: Internet
Author: User
File_get_contents captures garbled web pages. Sometimes, garbled characters occur when you use the file_get_contents () function to Capture webpages. There are two possible causes for garbled characters: encoding and Gzip on the target page. The encoding problem is good. sometimes, garbled characters occur when the file_get_contents () function is used to capture webpages. There are two possible causes for garbled characters: encoding and Gzip on the target page.

Encoding problems are easy to solve, just encode the captured content ($ content = iconv ("GBK", "UTF-8 // IGNORE", $ content );), here we will discuss how to capture the Gzip page. How to judge? Content-Encoding: gzip indicates that the Content is compressed by GZIP. If you use FireBug, you can see that gzip is not enabled on the page. The following shows the header information of my blog with firebug. Gzip is enabled.

Original request header information: Accepttext/html, application/xhtml + xml, application/xml; q = 0.9, */*; q = 0.8Accept-Encodinggzip, deflateAccept-Languagezh-cn, zh; q = 0.8, en-us; q = 0.5, en; q = 0.3Connectionkeep-aliveCookie _ utma = encrypt; _ utmz = 225240837.1326850415.887.3.utmcsr = google | utmccn =) | utmcmd = organic | utmctr = % E4 % BB % E4 % BD % 95% E9 % A1 % B9 % E7 % 9B % AE % E9 % 83% BD % E4 % B8 % 8D % E4 % BC % 9A % E9 % 82% A3 % E4 % B9 % 88% E7 % AE % 80% E5 % 8D % 95% 20 site % 3Awww.bkjia.com; PHPSESSID = 888mj4jwp8s0m7s0frre3ovc7; _ utmc = 225240837; _ utmb = runtime-AgentMozilla/5.0 (Windows NT 5.1; rv: 12.0) Gecko/20100101 Firefox/12.0

The following describes some solutions:

1. Use the built-in zlib library

If the zlib library has been installed on the server, the following code can easily solve the garbled problem.

$data = file_get_contents("compress.zlib://".$url); 

2. use CURL to replace file_get_contents

Function curl_get ($ url, $ gzip = false) {$ curl = curl_init ($ url); curl_setopt ($ curl, CURLOPT_RETURNTRANSFER, 1); curl_setopt ($ curl, CURLOPT_CONNECTTIMEOUT, 10); if ($ gzip) curl_setopt ($ curl, CURLOPT_ENCODING, "gzip"); // The key here is $ content = curl_exec ($ curl); curl_close ($ curl ); return $ content ;}

3. use the gzip decompression function

function gzdecode($data) {   $len = strlen($data);   if ($len < 18 || strcmp(substr($data,0,2),"\x1f\x8b")) {     return null;  // Not GZIP format (See RFC 1952)   }   $method = ord(substr($data,2,1));  // Compression method   $flags  = ord(substr($data,3,1));  // Flags   if ($flags & 31 != $flags) {     // Reserved bits are set -- NOT ALLOWED by RFC 1952     return null;   }   // NOTE: $mtime may be negative (PHP integer limitations)   $mtime = unpack("V", substr($data,4,4));   $mtime = $mtime[1];   $xfl   = substr($data,8,1);   $os    = substr($data,8,1);   $headerlen = 10;   $extralen  = 0;   $extra     = "";   if ($flags & 4) {     // 2-byte length prefixed EXTRA data in header     if ($len - $headerlen - 2 < 8) {       return false;    // Invalid format     }     $extralen = unpack("v",substr($data,8,2));     $extralen = $extralen[1];     if ($len - $headerlen - 2 - $extralen < 8) {       return false;    // Invalid format     }     $extra = substr($data,10,$extralen);     $headerlen += 2 + $extralen;   }   $filenamelen = 0;   $filename = "";   if ($flags & 8) {     // C-style string file NAME data in header     if ($len - $headerlen - 1 < 8) {       return false;    // Invalid format     }     $filenamelen = strpos(substr($data,8+$extralen),chr(0));     if ($filenamelen === false || $len - $headerlen - $filenamelen - 1 < 8) {       return false;    // Invalid format     }     $filename = substr($data,$headerlen,$filenamelen);     $headerlen += $filenamelen + 1;   }   $commentlen = 0;   $comment = "";   if ($flags & 16) {     // C-style string COMMENT data in header     if ($len - $headerlen - 1 < 8) {       return false;    // Invalid format     }     $commentlen = strpos(substr($data,8+$extralen+$filenamelen),chr(0));     if ($commentlen === false || $len - $headerlen - $commentlen - 1 < 8) {       return false;    // Invalid header format     }     $comment = substr($data,$headerlen,$commentlen);     $headerlen += $commentlen + 1;   }   $headercrc = "";   if ($flags & 1) {     // 2-bytes (lowest order) of CRC32 on header present     if ($len - $headerlen - 2 < 8) {       return false;    // Invalid format     }     $calccrc = crc32(substr($data,0,$headerlen)) & 0xffff;     $headercrc = unpack("v", substr($data,$headerlen,2));     $headercrc = $headercrc[1];     if ($headercrc != $calccrc) {       return false;    // Bad header CRC     }     $headerlen += 2;   }   // GZIP FOOTER - These be negative due to PHP's limitations   $datacrc = unpack("V",substr($data,-8,4));   $datacrc = $datacrc[1];   $isize = unpack("V",substr($data,-4));   $isize = $isize[1];   // Perform the decompression:   $bodylen = $len-$headerlen-8;   if ($bodylen < 1) {     // This should never happen - IMPLEMENTATION BUG!     return null;   }   $body = substr($data,$headerlen,$bodylen);   $data = "";   if ($bodylen > 0) {     switch ($method) {       case 8:         // Currently the only supported compression method:         $data = gzinflate($body);         break;       default:         // Unknown compression method         return false;     }   } else {     // I'm not sure if zero-byte body content is allowed.     // Allow it for now...  Do nothing...   }   // Verifiy decompressed size and CRC32:   // NOTE: This may fail with large data sizes depending on how   //       PHP's integer limitations affect strlen() since $isize   //       may be negative for large sizes.   if ($isize != strlen($data) || crc32($data) != $datacrc) {     // Bad format!  Length or CRC doesn't match!     return false;   }   return $data; }

Usage:

$html=file_get_contents('http://www.bkjia.com/librarys/veda/');$html=gzdecode($html);

This article introduces these three methods to solve the garbled crawling problem caused by most gzip files.

The file_get_contents () function captures webpages with garbled characters. There are two possible causes for garbled characters: encoding and Gzip on the target page. Encoding problems...

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: info-contact@alibabacloud.com and provide relevant evidence. A staff member will contact you within 5 working days.

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.