PHP also play concurrency, skillfully using curl concurrency to reduce back-end access time

Description: I am from 3 blog posts

http://blog.csdn.net/zuiaituantuan/article/details/7048782

First, understand the curl multithreaded functions in PHP:

# Curl_multi_add_handle
# Curl_multi_close
# curl_multi_exec
# curl_multi_getcontent
# Curl_multi_info_read
# Curl_multi_init
# Curl_multi_remove_handle
# Curl_multi_select

In general, when you think about using these functions, you should obviously want to request multiple URLs at the same time, rather than a single request, or you might as well cycle to tune curl_exec.

The steps are summarized as follows:

First step: Call Curl_multi_init
Step two: Loop call Curl_multi_add_handle
In this step, it should be noted that the second parameter of Curl_multi_add_handle is a curl_init handle.
Step three: Keep calling Curl_multi_exec
Step Fourth: Loop call curl_multi_getcontent to get results as needed
Step Fifth: Call Curl_multi_remove_handle and call curl_close for each word handle
Step Sixth: Call Curl_multi_close

Here is a simple example of a search on the web, the author of which is called the example of dirty (I will explain why dirty later):
/*
Here's a quick and dirty example for Curl-multi to PHP, tested on PHP 5.0.0rc1 Cli/freebsd 5.2.1
*/

$connomains = Array (
"http://www.cnn.com/",
"http://www.canada.com/",
"http://www.yahoo.com/"
);

$MH = Curl_multi_init ();

foreach ($connomains as $i => $url) {
$conn [$i]=curl_init ($url);
curl_setopt ($conn [$i],curlopt_returntransfer,1);
Curl_multi_add_handle ($MH, $conn [$i]);
}

do {$n =curl_multi_exec ($MH, $active);} while ($active);

foreach ($connomains as $i => $url) {
$res [$i]=curl_multi_getcontent ($conn [$i]);
Curl_close ($conn [$i]);
}

Print_r ($res);



This is pretty much what the whole use process is, but the simple code has a fatal weakness, that is, in the part of Do loop, it is a dead loop throughout the URL request, and it can easily cause the CPU to occupy 100%.

Now we're going to improve it, and here we'll use a function with almost no documentation curl_multi_select, although C's Curl Library has a description of the Select, but the interface and usage in PHP are different from C.

Change the paragraph above to do the following:
do {
$MRC = Curl_multi_exec ($MH, $active);
while ($MRC = = Curlm_call_multi_perform);
while ($active and $MRC = = CURLM_OK) {
if (Curl_multi_select ($MH)!=-1) {
do {
$MRC = Curl_multi_exec ($MH, $active);
while ($MRC = = Curlm_call_multi_perform);
}
}

Because $active to wait for all the URL data to become false, so here used the return value of curl_multi_exec to judge whether there is data, when there is data, call Curl_multi_exec, There is no data at the moment, and the new data can be awakened to continue execution as soon as it enters the select phase. The advantage here is that the unnecessary consumption of the CPU is gone.

In addition: There are some details that may sometimes be encountered:

Control the timeout for each request by curl_setopt before Curl_multi_add_handle:
curl_setopt ($ch, Curlopt_timeout, $timeout);

Determine if the timeout or other errors are used before Curl_multi_getcontent: Curl_error ($conn [$i]);

Here I'm simply using the dirty example above (enough, not finding CPU usage 100%).

The kandian.com interface simulates concurrency by reading and writing data to the memcache. The relevant data and results are not posted because of the confidentiality relationship.

Simulated 3 times, the first 10 threads simultaneously request 1000 times, the second, 100 threads simultaneously request 1000 times, the third time, 1000 threads simultaneously requests 100 times (already quite laborious, dares not to set over 1000 multithreading).

It seems that Curl multithreading simulation concurrency has some limitations.

In addition, it is suspected that the results of multithreading delay may result in large errors compared to data discovery. The time between initialization and set is very small, the difference is in Get method, so can simply exclude this ~ ~ ~

http://log.dongsheng.org/2008/07/16/curl-multiple-handlers/

In general, PHP CURL is blocked running, that is, after creating a CURL request must wait for it to perform a successful or timeout to execute the next request, Curl_multi_* series of functions to make concurrent access successful, PHP documents on this function is not very detailed, use the following:

$requests = Array (' http://www.baidu.com ', ' http://www.google.com ');
$main    = Curl_multi_init ();
$results = Array ();
$errors  = Array ();
$info = Array ();
$count = count ($requests);
for ($i = 0; $i < $count; $i + +) 
{  
$handles [$i] = Curl_init ($requests [$i]);  
Var_dump ($requests [$i]);  
curl_setopt ($handles [$i], Curlopt_url, $requests [$i]);  
curl_setopt ($handles [$i], Curlopt_returntransfer, 1);  
Curl_multi_add_handle ($main, $handles [$i]);
}
$running = 0; 
do {  
curl_multi_exec ($main, $running);
} 
while ($running > 0); 
for ($i = 0; $i < $count; $i + +)
{  $results [] = Curl_multi_getcontent ($handles [$i]);  
$errors []  = Curl_error ($handles [$i]);  
$info []    = Curl_getinfo ($handles [$i]);  
Curl_multi_remove_handle ($main, $handles [$i]);
}
Curl_multi_close ($main);
Var_dump ($results);
Var_dump ($errors);
Var_dump ($info);  

http://www.searchtb.com/2010/12/using-multicurl-to-improve-performance.html

Foreword: in our usual procedure inevitably appears simultaneously accesses several interfaces the situation, usually we use curl to carry on the visit, usually is a single, the sequential access, if has 3 interfaces, each interface takes 500 milliseconds then our three interfaces will spend 1500 milliseconds, This problem is too painful to seriously affect the page access speed, there is no possibility of concurrent access to improve speed. Today is simple to say, using curl concurrency to improve the speed of page access, I hope we have more guidance. 1, the old Curl access mode and time-consuming statistics

<?php function Curl_fetch ($url, $timeout =3) {     
$ch = Curl_init ();     
curl_setopt ($ch, Curlopt_url, $url);     
curl_setopt ($ch, Curlopt_timeout, $timeout);     
curl_setopt ($ch, Curlopt_returntransfer, 1);     
$data = curl_exec ($ch);     
$errno = Curl_errno ($ch);     
if ($errno >0) {         
$data = false;     
}     
Curl_close ($ch);     
return $data; 
} 
function Microtime_float () 
{    
list ($usec, $sec) = Explode ("", Microtime ());    
Return ((float) $usec + (float) $sec); 
} 
$url _arr=array (      
"Taobao" => "http://www.taobao.com", "Sohu" => "http://www.sohu.com"      
,      
"sina" = > "http://www.sina.com.cn",      
);  
$time _start = Microtime_float ();  
$data =array ();  
foreach ($url _arr as $key => $val)  
{      
$data [$key]=curl_fetch ($val);  
}  
$time _end = Microtime_float ();  
$time = $time _end-$time _start;  
echo "time consuming: {$time}"; 
? > 

Time consuming: 0.614 seconds
2, curl concurrent access and time-consuming statistics

<?php function Curl_multi_fetch ($urlarr =array ()) {$result = $res = $ch =array ();     
$nch = 0;     
$MH = Curl_multi_init ();         
foreach ($urlarr as $nk => $url) {$timeout = 2;         
$ch [$nch] = Curl_init (); Curl_setopt_array ($ch [$nch], Array (curlopt_url => $url, Curlopt_header => false, curlopt         
_returntransfer => True, Curlopt_timeout => $timeout,));         
Curl_multi_add_handle ($MH, $ch [$nch]);     + + $nch;     
}/* Wait for performing request */do {$MRC = Curl_multi_exec ($MH, $running);       
while (Curlm_call_multi_perform = = $MRC); while ($running && $MRC = = CURLM_OK) {//wait for network if (Curl_multi_select ($MH, 0.5) >             
-1) {//pull in new data;             
do {$MRC = Curl_multi_exec ($MH, $running);         
while (Curlm_call_multi_perform = = $MRC); } if ($MRC!= curlm_OK) {error_log ("CURL Data error");     
}/* Get data */$nch = 0; foreach ($urlarr as $moudle => $node) {if ($err = Curl_error ($ch [$nch]) = = = ') {$res [$nch]=curl             _multi_getcontent ($ch [$nch]);         $result [$moudle]= $res [$nch];         
else {error_log ("curl error");         
} curl_multi_remove_handle ($MH, $ch [$nch]);         
Curl_close ($ch [$nch]);     
+ + $nch;     
} curl_multi_close ($MH); 
return $result; } $url _arr=array ("Taobao" => "http://www.taobao.com", "Sohu" => "http://www.sohu.com", "Sina" => "H 
Ttp://www.sina.com.cn ",);    
function microtime_float () {list ($usec, $sec) = Explode ("", Microtime ()); 
Return ((float) $usec + (float) $sec); 
} $time _start = Microtime_float (); 
$data =curl_multi_fetch ($url _arr); 
$time _end = Microtime_float (); 
$time = $time _end-$time _start; 
echo "time consuming: {$time}"; 
 ?>

Time consuming: 0.316 seconds
It's cool. The entire page accesses back-end interfaces at half the time saved
3, Curl related parameters
From: http://cn2.php.net/manual/en/ref.curl.php
Curl_close-close a Curl session
Curl_copy_handle-copy a curl handle along with all of its preferences
Curl_errno-return the last Error number
Curl_error-return A string containing the last error for the
Curl_exec-perform a Curl session
Curl_getinfo-get information regarding a specific transfer
Curl_init-initialize a Curl session
Curl_multi_add_handle-add a normal curl handle to a curl multi handle
Curl_multi_close-close a set of curl handles
Curl_multi_exec-run the sub-connections of the current curl handle
Curl_multi_getcontent-return the content of a curl handle if Curlopt_returntransfer is set
Curl_multi_info_read-get information about the current transfers
Curl_multi_init-returns a new Curl multi handle
Curl_multi_remove_handle-remove a multi handle from a set of curl handles
Curl_multi_select-wait for Curl_multi Connection
Curl_setopt_array-set multiple options for a curl transfer
Curl_setopt-set an option for a curl transfer
Curl_version-gets Curl Version Information

Performance in front-end development that's the thing. (iii) PHP's opcode cache

Performance in front-end development that thing (a) skillfully use Xdebug