Crawl source code-PHP crawl Web source problems

Crawl the General page no problem ·.
But recently to a site to get dizzy · Please help the gods to see what causes the failure. Thank you

Address: http://183.36.249.82/

The Library of our school.

I use the following code

However, this error can occur:

Server Error in '/' Application.

Runtime Error

The server I'm using is SAE. Also tried to use the functionality provided by the SAE fetch .

But unfortunately also error.

Error is: string (3) "" string "Internal Server Error"

The reasons for the multi-day study are puzzling. Solution. Thank you

Reply content:

Crawl the General page no problem ·.
But recently to a site to get dizzy · Please help the gods to see what causes the failure. Thank you

Address: http://183.36.249.82/

The Library of our school.

I use the following code

However, this error can occur:

Server Error in '/' Application.

Runtime Error

The server I'm using is SAE. Also tried to use the functionality provided by the SAE fetch .

But unfortunately also error.

Error is: string (3) "" string "Internal Server Error"

The reasons for the multi-day study are puzzling. Solution. Thank you

file_get_contentswhen using the crawl, it is common to set the HTTP context. Crawling Web pages is better with curl.

  
    array(        'method' => 'GET',        'header' => $header,        'timeout' => 10,        'protocol_version' => '1.1',    

Don't you see the Web site requires redirection? Did you just learn php? Swap with curl, file_get_contents can not catch this bar (obviously I was wrong, look at the answer of others know is OK, no culture is really scary AH).

Let me give you a reference link, I don't want to write code.
http://segmentfault.com/q/1010000000484804#a-1020000000484819
Ask the question person although ask is the question of Ruby, but people's PHP code estimate enough for your reference.

