To capture the content of a remote Webpage through a local proxy, the Code is as follows: {code ...} phpfile. the phpnginx access log is 15.196.206.102 [26Apr2014: 12: 04: 45 + 0800] www. bigxu. com200206300.241 & quot;-& quot; 15.196.206.102 is my... to capture the content of a remote Webpage through a local proxy, the Code is as follows:
array( 'proxy'=>'tcp://192.168.1.108:8087', 'request_fulluri '=>true, "method" => "GET", "timeout" => 2, ),);$context = stream_context_create($options);$fp = stream_socket_client("tcp://www.bigxu.com:80", $errno, $errstr, 30,STREAM_CLIENT_CONNECT,$context);// print_r(stream_context_get_options($fp)); exit;if (!$fp) { echo "$errstr ($errno)
\n";} else { fwrite($fp, "GET / HTTP/1.0\r\nHost: www.bigxu.com\r\nAccept: */*\r\n\r\n"); while (!feof($fp)) { echo fgets($fp, 1024); } fclose($fp);}?>
Php file. php
The nginx access log is:
15.196.206.102 [26/Apr/2014: 12: 04: 45 + 0800] http://www.bigxu.com/200 20630 0.241 "-""-"
15.196.206.102 is my local IP address.
$ Context does not play a role.
Proxy is absolutely usable.
Because $ context works through the following code:
$options = array( 'http'=>array( 'proxy'=>'tcp://192.168.1.108:8087', 'request_fulluri '=>true, "method" => "GET", "timeout" => 2, ), ); $context = stream_context_create($options); if ( $fp = fopen("http://www.bigxu.com", 'r', false, $context) ) { print "well done"; while (!feof($fp)) { echo fgets($fp, 1024); } }
Php file. php
The nginx access log for bigxu.com is:
8.35.201.32 [26/Apr/2014: 12: 03: 03 + 0800] http://www.bigxu.com/200 7070 0.122 "-" AppEngine-Google; (+ http://code.google.com/appengine; appid: s ~ Goagent0527 )"
8.35.201.32 is my proxy IP Address
For large crawler projects, I will definitely use stream_socket_client to connect to them. Let me take a look at this function. How can I use it wrong?
Reply content:
To capture the content of a remote Webpage through a local proxy, the Code is as follows:
array( 'proxy'=>'tcp://192.168.1.108:8087', 'request_fulluri '=>true, "method" => "GET", "timeout" => 2, ),);$context = stream_context_create($options);$fp = stream_socket_client("tcp://www.bigxu.com:80", $errno, $errstr, 30,STREAM_CLIENT_CONNECT,$context);// print_r(stream_context_get_options($fp)); exit;if (!$fp) { echo "$errstr ($errno)
\n";} else { fwrite($fp, "GET / HTTP/1.0\r\nHost: www.bigxu.com\r\nAccept: */*\r\n\r\n"); while (!feof($fp)) { echo fgets($fp, 1024); } fclose($fp);}?>
Php file. php
The nginx access log is:
15.196.206.102 [26/Apr/2014: 12: 04: 45 + 0800] http://www.bigxu.com/200 20630 0.241 "-""-"
15.196.206.102 is my local IP address.
$ Context does not play a role.
Proxy is absolutely usable.
Because $ context works through the following code:
$options = array( 'http'=>array( 'proxy'=>'tcp://192.168.1.108:8087', 'request_fulluri '=>true, "method" => "GET", "timeout" => 2, ), ); $context = stream_context_create($options); if ( $fp = fopen("http://www.bigxu.com", 'r', false, $context) ) { print "well done"; while (!feof($fp)) { echo fgets($fp, 1024); } }
Php file. php
The nginx access log for bigxu.com is:
8.35.201.32 [26/Apr/2014: 12: 03: 03 + 0800] http://www.bigxu.com/200 7070 0.122 "-" AppEngine-Google; (+ http://code.google.com/appengine; appid: s ~ Goagent0527 )"
8.35.201.32 is my proxy IP Address
For large crawler projects, I will definitely use stream_socket_client to connect to them. Let me take a look at this function. How can I use it wrong?
It's hard to imagine that I found the answer on a Japanese website, so don't resist Japanese knowledge at will. Haha! After testing, it can be fully implemented. You can see:
Address: http://pe5974.sakura.ne.jp/contents/proxy-https.php
When it comes to collection, we should first think of the php curl Function
The best way is to simulate crawlers (such as Baidu spider crawlers or google spider crawlers) and support proxy configuration.
Simulating browser head requests through crawlers, there is nothing to catch (theoretically, as long as the data requested through the browser can be captured, whether it is to log on or not to log on)
I'm not sure if there is a problem with this proxy. Can I configure a proxy of my own?
Or follow up? Stream_socket_client is empty. Continue.