Phpcurl batch processing-controllable concurrency asynchronous

Phpcurl batch processing-controllable concurrency asynchronous code in the php Manual: $ mrccurl_multi_init (); send a request ....... $ activenull; do {$ mrccurl_multi_exec ($ mh, $ active);} while ($ mrcCURLM_CALL _ php curl batch processing-controllable concurrency asynchronous
There is a piece of code in the php manual:

$ Mrc = curl_multi_init (); // send a request ....... $ active = null; do {$ mrc = curl_multi_exec ($ mh, $ active);} while ($ mrc = CURLM_CALL_MULTI_PERFORM ); while ($ active & $ mrc = CURLM_ OK) {if (curl_multi_select ($ mh )! =-1) {do {$ mrc = curl_multi_exec ($ mh, $ active);} while ($ mrc = CURLM_CALL_MULTI_PERFORM);} // the result returned by the request is as follows:

This section do... while is used to control concurrency. The effect is to finish all the requests of $ mrc and then execute the response result.

However, if I have 1000 requests, the curl batch processing will concurrently process 1000 requests, which is obviously unreasonable. Therefore, we should control the number of concurrent requests and add the remaining connections to the request queue:
Reference: http://www.onlineaspect.com/2009/01/26/how-to-use-curl_multi-without-blocking/

$ Mh = curl_multi_init (); $ ch = array (); $ chunck = 10; // concurrency control count $ all = count ($ urls ); // all Request url arrays $ chunck = $ all> $ chunck? $ Chunck: $ all; $ options = array (CURLOPT_HEADER => FALSE, CURLOPT_RETURNTRANSFER => TRUE, CURLOPT_FOLLOWLOCATION => TRUE, CURLOPT_MAXREDIRS => 5, CURLOPT_USERAGENT => 'mozilla/5.0 (Windows NT 6.1; rv: 6.0) Gecko/20100101 Firefox/8080'); for ($ I = 0; $ I <$ chunck; $ I ++) {$ ch [$ I] = curl_init (); curl_setopt ($ ch [$ I], CURLOPT_URL, $ urls [$ I]); curl_setopt_array ($ ch [$ I], $ options); curl_multi_add_handle ($ mh, $ ch [$ I]);} do {while ($ Execrun = curl_multi_exec ($ mh, $ running) = CURLM_CALL_MULTI_PERFORM); if ($ execrun! = CURLM_ OK) break; // a request was just completed -- find out which one while ($ done = curl_multi_info_read ($ mh )) {// Obtain the index $ index = array_search ($ done ['handle'], $ ch) of the returned url in the urls array ); $ info = curl_getinfo ($ done ['handle']); if ($ info ['http _ Code'] = 200) {$ output = curl_multi_getcontent ($ ch [$ index]); // request successful. process output using the callback function. $ save_path = polici.'.txt '; // data storage path file_put_contents ($ save_path, $ output); // start a new request (it's important to do this before removing the old one) $ next = $ I ++; // increment I $ ch [$ next] = curl_init (); $ options [CURLOPT_URL] = $ urls [$ next]; // add the next request to the queue curl_setopt_array ($ ch [$ next], $ options); curl_multi_add_handle ($ mh, $ ch [$ next]); // remove the curl handle that just completed curl_multi_remove_handle ($ mh, $ done ['handle']);} else {// request failed. add error handling .}}} while ($ running); curl_multi_close ($ mh );

It works well and has no side effects, controllable concurrency, and many applications. let's take full advantage of your imagination.