It's all about a Unix-only server-compatible multi-process, and here's a multi-course that's compatible with both Windows and UNIX (in this case, the following curl is actually implemented by IO multiplexing).
A typical example of multithreading by extension is the ability of Curl,curl to crawl Web pages that support multithreading.
This part is too abstract, so I first give a curl to crawl multiple Web content in parallel a sub-assembly class. This class is actually very practical,
A detailed analysis of the internal implementation of these functions is described in the next tutorial.
You may not be able to understand this class well, and there are many wrong examples of Php curl on the official homepage, which tells about its internal mechanism
, you will be able to understand.
Look at the code first:
classhttp_multirequest{//List of URLs to crawl in parallel Private$urls =Array (); //Options for Curl Private$options; //constructor Functionfunction __construct ($options =Array ()) { $ This-setoptions ($options); } //set up a URL listfunction Seturls ($urls) {$ This->urls =$urls; return$ This; } //Setting Optionsfunction SetOptions ($options) {$options [Curlopt_returntransfer]=1; if(Isset ($options ['Http_post']) {curl_setopt ($ch, Curlopt_post,1); curl_setopt ($ch, Curlopt_postfields, $options ['Http_post']); Unset ($options ['Http_post']); } if(!isset ($options [curlopt_useragent]) {$options [curlopt_useragent]='mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1;)'; } if(!isset ($options [curlopt_followlocation]) {$options [curlopt_followlocation]=1; } if(!isset ($options [Curlopt_header]) {$options [Curlopt_header]=0; } $ This->options =$options; } //Crawl all content in parallelfunction exec () {if(Empty ($ This->urls) | | !is_array ($ This-URLs)) { return false; } $curl= $data =Array (); $MH=Curl_multi_init (); foreach($ This->urls as$k =$v) {$curl [$k]= $ This-AddHandle ($MH, $v); } $ This-Execmulithandle ($MH); foreach($ This->urls as$k =$v) {$data [$k]=curl_multi_getcontent ($curl [$k]); Curl_multi_remove_handle ($MH, $curl [$k]); } curl_multi_close ($MH); return$data; } //Crawl only one page of content. function Execone ($url) {if(Empty ($url)) {return false; } $ch=Curl_init ($url); $ This-setoneoption ($ch); $content=curl_exec ($ch); Curl_close ($ch); return$content; } //an intrinsic function that sets an option for a handle Privatefunction Setoneoption ($ch) {Curl_setopt_array ($ch, $ This-options); } //add a new parallel fetch handle Privatefunction AddHandle ($MH, $url) {$ch=Curl_init ($url); $ This-setoneoption ($ch); Curl_multi_add_handle ($MH, $ch); return$ch; } //Parallel Execution (This is a common mistake, and I'm still using this notation here//downloading a small file can cause the cup to occupy 100%, and the loop will run more than 100,000 times//This is a typical error that does not understand the principle. This error is quite common in the official PHP documentation. ) Privatefunction ExecMulitHandle2 ($mh) {$i=0; $running=NULL; Do{curl_multi_exec ($MH, $running); $i++; } while($running >0); //Var_dump ($i); } //It should be written like this. Privatefunction Execmulithandle ($mh) {$i=0; Do{$MRC = Curl_multi_exec ($MH, $active); $i + +;} while($MRC = =curlm_call_multi_perform); while($active && $MRC = =CURLM_OK) { if(Curl_multi_select ($MH)! =-1) { Do{$MRC = Curl_multi_exec ($MH, $active); $i + +;} while($MRC = =curlm_call_multi_perform); } $i++; } //Var_dump ($i); }}
See the last one of the most commented functions, this error may not be easy to detect when debugging, because the program is completely normal, but under the production server, will immediately cause a crash effect.
Explain why this cannot be done and must be analyzed from the perspective of the internal implementation of the C language. This section will be placed in the next tutorial (PHP Advanced Programming – Single threaded implementation to crawl Web pages in parallel). But not through the C language to express the principle, but through the PHP
This class, in fact, is very simple implementation of the previous we cost 4 tutorials, and is Dickens to implement the multi-threaded crawl Web page function. In the implementation of pure PHP, we can only use a background service to better implement, but when you use the operating system interface Language C language, this implementation of course is more simple, flexible, efficient.
At the same time to crawl a few web pages such a simple thing, actually involved in the bottom of a lot of things, for many halfway decent PHP programmers, may not like to talk about multithreading this thing, in-depth on the operating system, shallow point is parallel running several "programs." However, many times, multithreading is essential, such as to write a fast crawler, often waste Dickens. However, PHP programmers should now thank Curl for the extension, so that you do not have to use your less proficient python to write crawlers, for a medium-sized crawler, there is this internal multi-threading, is enough.
Finally, here is an example of a test for the above class:
$urls = Array ("http://baidu.com","http://baidu.com","http://baidu.com","http://baidu.com","http://baidu.com","http://baidu.com","http://www.google.com","http://www.sina.com.cn",); $m=Newhttp_multirequest (); $t= Microtime (true); $m-Seturls ($urls);//parallel fetch (parallel fetch):$data = $mexec (); $parallel _time= Microtime (true) -$t, Echo $parallel _time."\ n"; $t= Microtime (true); //serial fetch (serial fetch):foreach($urls as$url) {$data []= $mExecone ($url);} $serial _time= Microtime (true) -$t, Echo $serial _time."\ n";
PHP Multi-process (4): Internal Multi-process