Detailed description of concurrent simulation using curl multithreading. First, let's take a look at the curl multithreading function in php: Copy the code as follows: # curl_multi_add_handle # curl_multi_close # curl_multi_exec # curl_multi_getcontent # curl_multi_ I
First, let's take a look at the curl multithreading function in php:
The code is as follows:
# Curl_multi_add_handle
# Curl_multi_close
# Curl_multi_exec
# Curl_multi_getcontent
# Curl_multi_info_read
# Curl_multi_init
# Curl_multi_remove_handle
# Curl_multi_select
Generally, when you want to use these functions, it is obvious that you need to request multiple URLs at the same time, instead of one request in sequence. Otherwise, it is better to call curl_exec cyclically.
The steps are summarized as follows:
Step 1: Call curl_multi_init
Step 2: call curl_multi_add_handle cyclically
Note that the second parameter of curl_multi_add_handle is the sub-handle from curl_init.
Step 3: continuously call curl_multi_exec
Step 4: call curl_multi_getcontent cyclically to obtain the result as needed
Step 5: call curl_multi_remove_handle and call curl_close for each handle.
Step 6: call curl_multi_close
Here is a simple online search example by dirty (I will explain why dirty later ):
The code is as follows:
/*
Here's a quick and dirty example for curl-multi from PHP, tested on PHP 5.0.0RC1 CLI/FreeBSD 5.2.1
*/
$ Connomains = array (
"Http://www.cnn.com /",
"Http://www.canada.com /",
"Http://www.yahoo.com /"
);
$ Mh = curl_multi_init ();
Foreach ($ connomains as $ I => $ url ){
$ Conn [$ I] = curl_init ($ url );
Curl_setopt ($ conn [$ I], CURLOPT_RETURNTRANSFER, 1 );
Curl_multi_add_handle ($ mh, $ conn [$ I]);
}
Do {$ n = curl_multi_exec ($ mh, $ active);} while ($ active );
Foreach ($ connomains as $ I => $ url ){
$ Res [$ I] = curl_multi_getcontent ($ conn [$ I]);
Curl_close ($ conn [$ I]);
}
Print_r ($ res );
This is almost the same throughout the use process. However, this simple code has a fatal weakness, that is, in the do loop section, it is an endless loop during the whole url request, it can easily cause 100% CPU usage.
Now let's improve it. here we need to use a function named curl_multi_select, which has almost no documentation. Although the C curl Library describes select, the interfaces and usage in php are indeed different from those in C.
Change the do section above to the following:
The code is as follows:
Do {
$ Mrc = curl_multi_exec ($ mh, $ active );
} While ($ mrc = CURLM_CALL_MULTI_PERFORM );
While ($ active and $ mrc = CURLM_ OK ){
If (curl_multi_select ($ mh )! =-1 ){
Do {
$ Mrc = curl_multi_exec ($ mh, $ active );
} While ($ mrc = CURLM_CALL_MULTI_PERFORM );
}
}
Because $ active changes to false only after all url data is accepted, the returned value of curl_multi_exec is used to determine whether there is any data. when there is data, curl_multi_exec is called continuously, if no data is available for the moment, the select phase is started, and the new data can be awakened to continue execution. The advantage here is that there is no unnecessary CPU consumption.
In addition, there are some details that may sometimes be encountered:
Control the timeout time of each request. use curl_setopt before curl_multi_add_handle:
Curl_setopt ($ ch, CURLOPT_TIMEOUT, $ timeout );
Determine whether the request times out or other errors. use curl_error ($ conn [$ I]) before curl_multi_getcontent.
Here I am simply using the preceding dirty example (enough, no 100% cpu usage is found ).
Simulate concurrency for an interface of "aspect" (kandian.com). The function is to read and write data to memcache. Because of the confidentiality, the relevant data and results will not be posted.
Simulation 3 times, the first 10 threads simultaneously request 1000 times, the second, 100 threads simultaneously request 1000 times, the third, 1000 threads simultaneously request 100 times (already quite laborious, do not dare to set more than 1000 multithreading ).
It seems that curl multi-thread concurrency simulation is still limited.
In addition, it is suspected that the results may be large errors due to the multi-thread delay, and the comparison data is found. The time used for initialization and set is not significant, and the difference lies in the get method. Therefore, this can be simply ruled out ~~~
The code for the curl multithreading function in php is as follows: # curl_multi_add_handle # curl_multi_close # curl_multi_exec # curl_multi_getcontent # curl_multi_ I...
