Windows multi-process plus coprocessor concurrency mode

I haven't updated my blog for a long time. It's time to sort out some of the most recent projects and learn about Python. As the title shows, today it's time to talk about Windows multi-process plus co-concurrent mode. In fact, on the internet is quite a lot of Linux under the multi-process and co-concurrent mode, the Linux itself support for Python is better. But because my development environment is windows, and the information on the Internet is still a little bit less, but after some toss, it is also made. Don't say much nonsense, put the code first:

#Coding=utf-8#Windows Multi-process plus coprocessor concurrency mode#Monkey patch into Gevent fromGeventImportMonkeymonkey.patch_all ()#To import the Federation pool fromGevent.poolImportPoolImportUrllib2#importing multi-process modulesImportMultiprocessingImport Time#define a way to crawl a microblog web pagedefHTML (url=u'http://weibo.com/'):    #lock mechanism with multiple processes to prevent continuous printingLock =multiprocessing. Lock ()Try: H=urllib2.urlopen (URL). Read ()except: Lock.acquire ()PrintU'Connection Error'lock.release ()Else: Lock.acquire ()PrintU' Done'lock.release ()#define a co-concurrent method (using the gevent's pool)defA (num): Pool= Pool (100)    #the pool map method allows you to customize the number of concurrent times, where you can customize the number of concurrent crawls of a Web page, and the first parameter is the function to be executed    #The second parameter can be interpreted as the number of times a concurrency parameter is requiredPool.map (HTML, [u'http://weibo.com/'  forIinchxrange (num)]) Pool.kill () Pool.join ( )#This is a more critical approach, that is, the concurrency pattern of the co-Chenggado processdefb (num): t= []    #set up 10 processes to parallel the process as method a     forIinchXrange (10): P= multiprocessing. Process (Target=a, args=(num,)) P.start () T.append (p) foreachinchT:each.join ()#comparing the respective concurrency states of the multi-process plus and the purely co-routinesif __name__=='__main__':    PrintU'Method B Starts the timing:'Start=Time.time () b (10)    PrintU'Method B takes a total of%f seconds'% (Time.time ()-start)PrintU'method A starts timing:'Start=Time.time () A (100)    PrintU'method A takes a total of%f seconds'% (Time.time ()-start)

The above code comment is very clear, the following paste the results of the implementation of it:

As can be seen from the above results, the implementation of the Chenggado process of method B, multi-core CPU can be fully utilized, this is simply the use of the process is not possible, this is a big advantage of many processes. Because I experiment here is 100 times concurrency, so actually the effect is not very obvious, but when you want to instantaneous concurrency 1000 or 10,000, the gap can be revealed, obviously, the Chenggado process of the way is more bull fork, this model fully combine the advantages of the two.

Here, if the reader is not familiar with the Python module and the multi-process module, it is self-Baidu a bit. After all, I really understand that it is their own things ah. In fact, I am also a Python rookie, if the above code there is any problem or better advice, but also hope that the great God to enlighten Ah!

Windows multi-process plus coprocessor concurrency mode