php curl 代理 抓取資料

來源:互聯網
上載者:User
<?phpdefine ( 'IS_PROXY', true ); //是否啟用代理function async_get_url($url_array, $wait_usec = 0){    if (!is_array($url_array))        return false;    $wait_usec = intval($wait_usec);    $data    = array();    $handle  = array();    $running = 0;    $mh = curl_multi_init(); // 開啟多線程    $i = 0;    foreach($url_array as $url) {        $ch = curl_init();if (IS_PROXY) {//以下代碼設定Proxy 伺服器//Proxy 伺服器地址http://www.cnproxy.com/proxy1.html  !!Hong Kong, China的速度比較好curl_setopt ($ch, CURLOPT_PROXY,'110.4.12.170:80' );}        curl_setopt($ch, CURLOPT_URL, $url);        curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1); // return don't print        curl_setopt($ch, CURLOPT_TIMEOUT, 30); //設定逾時時間        curl_setopt($ch, CURLOPT_USERAGENT, 'Mozilla/4.0 (compatible; MSIE 5.01; Windows NT 5.0)');        curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1); // 302 redirect        curl_setopt($ch, CURLOPT_MAXREDIRS, 7); //HTTp定向層級        curl_multi_add_handle($mh, $ch); // 把 curl resource 放進 multi curl handler 裡        $handle[$i++] = $ch;    }    /* 執行 */do {$mrc = curl_multi_exec($mh, $running);if ($wait_usec > 0) /* 每個 connect 要間隔多久 */            usleep($wait_usec); // 250000 = 0.25 sec} while ($mrc == CURLM_CALL_MULTI_PERFORM);while ($running && $mrc == CURLM_OK) {if (curl_multi_select($mh) != -1) {do {$mrc = curl_multi_exec($mh, $running);} while ($mrc == CURLM_CALL_MULTI_PERFORM);}}    /* 讀取資料 */    foreach($handle as $i => $ch) {        $content  = curl_multi_getcontent($ch);        $data[$i] = (curl_errno($ch) == 0) ? $content : false;    }    /* 移除 handle*/    foreach($handle as $ch) {        curl_multi_remove_handle($mh, $ch);    }    curl_multi_close($mh);    return $data;}$urls = array('http://map.baidu.com');$re = async_get_url($urls); echo $re[0];?> 

轉自:http://hi.baidu.com/ylj798/item/322468136b70bc433a176efc

http://www.oschina.net/code/snippet_80475_6689

curl入門:http://www.chinaz.com/program/2010/0119/104346_5.shtml

免費Proxy 伺服器:http://www.cnproxy.com/proxy1.html

相關文章

聯繫我們

該頁面正文內容均來源於網絡整理,並不代表阿里雲官方的觀點,該頁面所提到的產品和服務也與阿里云無關,如果該頁面內容對您造成了困擾,歡迎寫郵件給我們,收到郵件我們將在5個工作日內處理。

如果您發現本社區中有涉嫌抄襲的內容,歡迎發送郵件至: info-contact@alibabacloud.com 進行舉報並提供相關證據,工作人員會在 5 個工作天內聯絡您,一經查實,本站將立刻刪除涉嫌侵權內容。

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.