PHP Multi-threaded Batch acquisition download image implementation code

Multi-Threading using curl, in addition to curl can set the request time, encountered a very slow URL resources, can be determined to give up, so that there is no blocking, in addition to multi-threaded requests, the efficiency should be higher /** * Curl Multithreading * @param array $array parallel URLs * @param int $timeout time-out * @return Mix */ Public Function Curl_http ($array, $timeout = ' 15 ') { $res = Array (); $MH = Curl_multi_init ();//Create Multiple Curl handles foreach ($array as $k = = $url) { $conn [$k]=curl_init ($url);//initialization curl_setopt ($conn [$k], curlopt_timeout, $timeout);//Set time-out curl_setopt ($conn [$k], Curlopt_useragent, ' mozilla/5.0 (compatible; MSIE 5.01; Windows NT 5.0) '); curl_setopt ($conn [$k], Curlopt_maxredirs, 7);//http orientation level, 7 maximum curl_setopt ($conn [$k], Curlopt_header, false);//Do not HEADER, add block efficiency curl_setopt ($conn [$k], curlopt_followlocation, 1); 302 redirect curl_setopt ($conn [$k], curlopt_returntransfer,1);//requires the result to be a string and output to the screen curl_setopt ($conn [$k], Curlopt_httpget, true); Curl_multi_add_handle ($MH, $conn [$k]); } To prevent the death cycle from killing the CPU. This is based on the online wording do { $MRC = Curl_multi_exec ($MH, $active);//When no data, active=true } while ($MRC = = Curlm_call_multi_perform);//When data is being accepted while ($active and $MRC = = CURLM_OK) {//when there is no data or when a request is paused, active=true if (Curl_multi_select ($MH)! =-1) { do { $MRC = Curl_multi_exec ($MH, $active); } while ($MRC = = Curlm_call_multi_perform); } } foreach ($array as $k = = $url) { if (!curl_errno ($conn [$k])) { $data [$k]=curl_multi_getcontent ($conn [$k]);//data converted to array $header [$k]=curl_getinfo ($conn [$k]);//return HTTP header information Curl_close ($conn [$k]);//Closing the handle Curl_multi_remove_handle ($MH, $conn [$k]); Freeing resources }else{ Unset ($k, $url); } } Curl_multi_close ($MH); return $data; } Parameter Receive $callback = $_get[' callback '); $hrefs = $_get[' hrefs '); $urlarray = Explode (', ', trim ($hrefs, ', ')); $date = Date (' Ymd ', Time ()); Instantiation of $img = new Httpimg (); $stime = $img->getmicrotime ();//Start time $data = $img->curl_http ($urlarray, ' 20 ');//List data mkdir ('./img/'. $date, 0777); foreach ((array) $data as $k = + $v) { Preg_match_all ("/(HREF|SRC) = ([" | ']?) ([^ "' >]+. (jpg|png| Png| Jpg|gif) \\2/i ", $v, $matches [$k]); if (count ($matches [$k][3]) >0) { $dataimg = $img->curl_http ($matches [$k][3], ' 20 ');//All picture data binary $j = 0; foreach ((array) $dataimg as $kk = + $vv) { if ($vv! = ") { $rand = rand (1000,9999); $basename = Time (). " _ ". $rand.". jpg;//files saved in JPG format $fname = './img/'. $date. " /"." $basename "; File_put_contents ($fname, $VV); $j + +; echo "Create section". $j. " Photo "." $fname "." "; }else{ Unset ($kk, $VV); } } }else{ Unset ($matches); } } $etime = $img->getmicrotime ();//End time echo "Spents". ($etime-$stime). " Seconds "; Exit Copy Code

Multithreading, download images, PHP

