PHP multi-threaded Batch collection download beautiful pictures of the implementation code

Source: Internet
Author: User
Tags foreach date array curl header rand return

Use Curl multithreading, in addition curl can set the request time, encountered a very slow URL resources, can decisively give up, so there is no blocking, there are multiple threads request, efficiency should be higher

Here is the code implementation:

 
 
  1. /**
  2. * Curl Multithreading
  3. * @author http://www.lai18.com
  4. * @param array $array parallel URLs
  5. * @param int $timeout timeout
  6. * @return Mix
  7. */
  8. Public Function Curl_http ($array, $timeout = ' 15 ') {
  9. $res = Array ();
  10. $MH = Curl_multi_init ();//Create Multiple Curl handles
  11. foreach ($array as $k => $url) {
  12. $conn [$k]=curl_init ($url);//initialization
  13. curl_setopt ($conn [$k], curlopt_timeout, $timeout);/Set timeout time
  14. curl_setopt ($conn [$k], Curlopt_useragent, ' mozilla/5.0 (compatible; MSIE 5.01; Windows NT 5.0);
  15. curl_setopt ($conn [$k], Curlopt_maxredirs, 7)//http orientation level, 7 highest
  16. curl_setopt ($conn [$k], Curlopt_header, false);/No HEADER here, add block efficiency
  17. curl_setopt ($conn [$k], curlopt_followlocation, 1); 302 redirect
  18. curl_setopt ($conn [$k], curlopt_returntransfer,1)//requires the result to be a string and output to the screen
  19. curl_setopt ($conn [$k], Curlopt_httpget, true);
  20. Curl_multi_add_handle ($MH, $conn [$k]);
  21. }
  22. Preventing dead loops from dying the CPU this paragraph is based on the online wording
  23. do {
  24. $MRC = Curl_multi_exec ($MH, $active);//When there is no data, active=true
  25. while ($MRC = = Curlm_call_multi_perform);////When data is being accepted
  26. while ($active and $MRC = = CURLM_OK) {//when there is no data or when the request is paused, active=true
  27. if (Curl_multi_select ($MH)!=-1) {
  28. do {
  29. $MRC = Curl_multi_exec ($MH, $active);
  30. while ($MRC = = Curlm_call_multi_perform);
  31. }
  32. }
  33. foreach ($array as $k => $url) {
  34. if (!curl_errno ($conn [$k])) {
  35. $data [$k]=curl_multi_getcontent ($conn [$k]);//data converted to array
  36. $header [$k]=curl_getinfo ($conn [$k]);//return HTTP header information
  37. Curl_close ($conn [$k]);//close handle
  38. Curl_multi_remove_handle ($MH, $conn [$k]); Releasing resources
  39. }else{
  40. Unset ($k, $url);
  41. }
  42. }
  43. Curl_multi_close ($MH);
  44. return $data;
  45. }
  46. Parameter Receive
  47. $callback = $_get[' callback '];
  48. $hrefs = $_get[' HREFs '];
  49. $urlarray = Explode (', ', trim ($hrefs, ', '));
  50. $date = Date (' Ymd ', Time ());
  51. Instantiation of
  52. $img = new Httpimg ();
  53. $stime = $img->getmicrotime ()//Start time
  54. $data = $img->curl_http ($urlarray, ' 20 ');//List data
  55. mkdir ('./img/'. $date, 0777);
  56. foreach ((array) $data as $k => $v) {
  57. Preg_match_all ("/(HREFSRC) = (["]?) ([^ "' >]+.] (jpgpngpngjpggif)) \2/i ", $v, $matches [$k]);
  58. if (count ($matches [$k][3]) >0) {
  59. $dataimg = $img->curl_http ($matches [$k][3], ' 20 ');//All picture data binary
  60. $j = 0;
  61. foreach ((array) $dataimg as $kk => $vv) {
  62. if ($vv!= ') {
  63. $rand = rand (1000,9999);
  64. $basename = Time (). _ ". $rand.". jpg;//files saved in JPG format
  65. $fname = './img/'. $date. " /"." $basename ";
  66. File_put_contents ($fname, $VV);
  67. $j + +;
  68. echo "Create the first". $j. " Picture "." $fname "." <br/> ";
  69. }else{
  70. Unset ($kk, $VV);
  71. }
  72. }
  73. }else{
  74. Unset ($matches);
  75. }
  76. }
  77. $etime = $img->getmicrotime ()/End time
  78. echo "Spents". ($etime-$stime). " Seconds ";
  79. Exit



Related Article

Alibaba Cloud 10 Year Anniversary

With You, We are Shaping a Digital World, 2009-2019

Learn more >

Apsara Conference 2019

The Rise of Data Intelligence, September 25th - 27th, Hangzhou, China

Learn more >

Alibaba Cloud Free Trial

Learn and experience the power of Alibaba Cloud with a free trial worth $300-1200 USD

Learn more >

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: info-contact@alibabacloud.com and provide relevant evidence. A staff member will contact you within 5 working days.