Solve the high CPU load caused by phpcurl_multi batch processing

Today, when using the curl_multi function to obtain some Internet content, I found that as long as the curl_multi function is run, my cpu usage is very high, later, I came to a webmaster and shared my solutions to this problem. I hope to help you... today, when using the curl_multi function to obtain some Internet content, I found that as long as the curl_multi function is run, my cpu usage is very high, later, I came to a webmaster and shared my solutions to this problem. I hope to help you.

The simple cURL processing is as follows:

$ch = curl_init(); curl_setopt($ch, CURLOPT_URL, 'http://www.phprm.com'); curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE); $con = curl_exec($ch); curl_close($ch);

CURL also provides batch processing sessions. below are the functions related to cURL batch processing:

Curl_multi_init-return a new cURL batch handle

Curl_multi_add_handle-add a separate curl handle to the curl batch processing session

Curl_multi_exec-parse a cURL batch handle

Curl_multi_getcontent-if CURLOPT_RETURNTRANSFER is set, the obtained output text stream is returned.

Curl_multi_select-wait for active connections in all cURL batch processing

Curl_multi_info_read-obtains the transmission information of the currently resolved cURL.

Curl_multi_remove_handle-remove a handle resource from the curl batch processing handle.

Curl_multi_close-close a group of cURL handles

The following example shows how to use curl multi batch processing. the code is as follows:

/*** CURL multi batch processing ** @ author mckee * @ link http://www.phprm.com **/$ Url_array = array (' http://www.phpfensi.net/ ',' http://www.phpfensi.net/ Php/627.html ',' http://www.phpfensi.net/ Php/258.html '); $ handles = $ contents = array (); // initialize the curl multi Object $ mh = curl_multi_init (); // add a curl batch processing session foreach ($ url_array as $ key =>$ url) {$ handles [$ key] = curl_init ($ url ); curl_setopt ($ handles [$ key], CURLOPT_RETURNTRANSFER, 1); curl_setopt ($ handles [$ key], CURLOPT_TIMEOUT, 10); curl_multi_add_handle ($ mh, $ handles [$ key]);} // open-source code phprm.com // ============================== execute the batch processing handle ======== ======================== =========$ Active = null; do {$ mrc = curl_multi_exec ($ mh, $ active) ;}while ($ mrc == CURLM_CALL_MULTI_PERFORM ); while ($ active and $ mrc = CURLM_ OK) {if (curl_multi_select ($ mh) =-1) {usleep (100 );} do {$ mrc = curl_multi_exec ($ mh, $ active);} while ($ mrc = CURLM_CALL_MULTI_PERFORM );} // ================================================ ==========================================/// get the batch content foreach ($ handles as $ I => $ c H) {$ content = curl_multi_getcontent ($ ch); $ contents [$ I] = curl_errno ($ ch) = 0? $ Content: '';} // remove the batch processing handle foreach ($ handles as $ ch) {curl_multi_remove_handle ($ mh, $ ch );} // Close the batch processing handle curl_multi_close ($ mh); print_r ($ contents );

The above section focuses on the section that executes batch processing. the common processing code is as follows:

do { $n=curl_multi_exec($mh,$active); } while ($active);

The CPU Loading is too high, because $ active is converted to false only after all the url data is accepted. Therefore, the returned value of curl_multi_exec is used to determine whether there is any data, when data exists, curl_multi_exec will be called continuously, and sleep will be performed without execution data. This will avoid CPU Loading 100%.

Permanent link:

Reprint at will! Include the article address.