PHP Multi-process (4): Internal Multi-process

Source: Internet
Author: User

It's all about a Unix-only server-compatible multi-process, and here's a multi-course that's compatible with both Windows and UNIX (in this case, the following curl is actually implemented by IO multiplexing).

A typical example of multithreading by extension is the ability of Curl,curl to crawl Web pages that support multithreading.

This part is too abstract, so I first give a curl to crawl multiple Web content in parallel a sub-assembly class. This class is actually very practical,

A detailed analysis of the internal implementation of these functions is described in the next tutorial.

You may not be able to understand this class well, and there are many wrong examples of Php curl on the official homepage, which tells about its internal mechanism

, you will be able to understand.

Look at the code first:

classhttp_multirequest{//List of URLs to crawl in parallel    Private$urls =Array (); //Options for Curl    Private$options; //constructor Functionfunction __construct ($options =Array ()) {        $ This-setoptions ($options); }     //set up a URL listfunction Seturls ($urls) {$ This->urls =$urls; return$ This; }     //Setting Optionsfunction SetOptions ($options) {$options [Curlopt_returntransfer]=1; if(Isset ($options ['Http_post']) {curl_setopt ($ch, Curlopt_post,1); curl_setopt ($ch, Curlopt_postfields, $options ['Http_post']); Unset ($options ['Http_post']); }         if(!isset ($options [curlopt_useragent]) {$options [curlopt_useragent]='mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1;)'; }         if(!isset ($options [curlopt_followlocation]) {$options [curlopt_followlocation]=1; }         if(!isset ($options [Curlopt_header]) {$options [Curlopt_header]=0; }        $ This->options =$options; }     //Crawl all content in parallelfunction exec () {if(Empty ($ This->urls) | | !is_array ($ This-URLs)) {            return false; } $curl= $data =Array (); $MH=Curl_multi_init (); foreach($ This->urls as$k =$v) {$curl [$k]= $ This-AddHandle ($MH, $v); }        $ This-Execmulithandle ($MH); foreach($ This->urls as$k =$v) {$data [$k]=curl_multi_getcontent ($curl [$k]);        Curl_multi_remove_handle ($MH, $curl [$k]);        } curl_multi_close ($MH); return$data; }     //Crawl only one page of content. function Execone ($url) {if(Empty ($url)) {return false; } $ch=Curl_init ($url); $ This-setoneoption ($ch); $content=curl_exec ($ch);        Curl_close ($ch); return$content; }     //an intrinsic function that sets an option for a handle    Privatefunction Setoneoption ($ch) {Curl_setopt_array ($ch, $ This-options); }     //add a new parallel fetch handle    Privatefunction AddHandle ($MH, $url) {$ch=Curl_init ($url); $ This-setoneoption ($ch);        Curl_multi_add_handle ($MH, $ch); return$ch; }     //Parallel Execution (This is a common mistake, and I'm still using this notation here//downloading a small file can cause the cup to occupy 100%, and the loop will run more than 100,000 times//This is a typical error that does not understand the principle. This error is quite common in the official PHP documentation. )    Privatefunction ExecMulitHandle2 ($mh) {$i=0; $running=NULL;  Do{curl_multi_exec ($MH, $running); $i++; }  while($running >0); //Var_dump ($i);    }     //It should be written like this.    Privatefunction Execmulithandle ($mh) {$i=0;  Do{$MRC = Curl_multi_exec ($MH, $active); $i + +;} while($MRC = =curlm_call_multi_perform);  while($active && $MRC = =CURLM_OK) {            if(Curl_multi_select ($MH)! =-1)            {                 Do{$MRC = Curl_multi_exec ($MH, $active); $i + +;} while($MRC = =curlm_call_multi_perform); } $i++; }        //Var_dump ($i);    }}

See the last one of the most commented functions, this error may not be easy to detect when debugging, because the program is completely normal, but under the production server, will immediately cause a crash effect.

Explain why this cannot be done and must be analyzed from the perspective of the internal implementation of the C language. This section will be placed in the next tutorial (PHP Advanced Programming – Single threaded implementation to crawl Web pages in parallel). But not through the C language to express the principle, but through the PHP

This class, in fact, is very simple implementation of the previous we cost 4 tutorials, and is Dickens to implement the multi-threaded crawl Web page function. In the implementation of pure PHP, we can only use a background service to better implement, but when you use the operating system interface Language C language, this implementation of course is more simple, flexible, efficient.

At the same time to crawl a few web pages such a simple thing, actually involved in the bottom of a lot of things, for many halfway decent PHP programmers, may not like to talk about multithreading this thing, in-depth on the operating system, shallow point is parallel running several "programs." However, many times, multithreading is essential, such as to write a fast crawler, often waste Dickens. However, PHP programmers should now thank Curl for the extension, so that you do not have to use your less proficient python to write crawlers, for a medium-sized crawler, there is this internal multi-threading, is enough.

Finally, here is an example of a test for the above class:

$urls = Array ("http://baidu.com","http://baidu.com","http://baidu.com","http://baidu.com","http://baidu.com","http://baidu.com","http://www.google.com","http://www.sina.com.cn",); $m=Newhttp_multirequest (); $t= Microtime (true); $m-Seturls ($urls);//parallel fetch (parallel fetch):$data = $mexec (); $parallel _time= Microtime (true) -$t, Echo $parallel _time."\ n"; $t= Microtime (true); //serial fetch (serial fetch):foreach($urls as$url) {$data []= $mExecone ($url);} $serial _time= Microtime (true) -$t, Echo $serial _time."\ n";

PHP Multi-process (4): Internal Multi-process

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: info-contact@alibabacloud.com and provide relevant evidence. A staff member will contact you within 5 working days.

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.