Importdef test1 (): print gevent.sleep (0) Printdef test2 (): print gevent.sleep (0) Print gevent.joinall ([ gevent.spawn (test1), gevent.spawn (test2),])
Explanation, the "gevent.spawn ()" method creates a new Greenlet-the-thread object and runs it. The "Gevent.joinall ()" Method waits for all incoming Greenlet to finish running before exiting, and this method can accept a "timeout" parameter to set the time-out, in seconds. Run the above program and execute the following sequence:
- Enter the Test1 first, print 12
- When encountering "Gevent.sleep (0)", Test1 is blocked, automatically switches to the co-test2, prints 56
- After the test2 is blocked, the test1 block is closed and automatically switches back to test1, printing 34
- When Test1 is finished running, the Test2 block is closed, and then automatically switches back to test2, printing 78
- The program exits after all the processes have been executed
So the output that the program runs down is:
12563478
Greenlet after a process has finished running, you must explicitly toggle it, or it will return its parent co-process. In Gevent, when a process is finished running, it automatically dispatches those unfinished processes.
import gevent import socket urls = [ " www.baidu.com ", " www.gevent.org ", " www.python.org " ]jobs = [ Gevent.spawn (socket.gethostbyname, url) for URL In Urls]gevent.joinall (jobs, timeout =5 print [job.value for job in Jobs]
We obtain the IP address of three websites by the association process separately, because the open remote address can cause IO blocking, so the gevent will automatically dispatch different co-processes. In addition, we can get the return value of the co-function by the "value" property of the co-process object.
Monkey Patch Monkey Patching
In fact, the above program to run the same time without the co-process is the same, is the sum of three web site open time. But in theory, the process is non-blocking, the running time should be equal to the longest site open time ah? In fact, this is because the Python standard database socket is blocking, DNS parsing can not be concurrent, including the same as the Urllib library, so in this case the use of the process is completely meaningless. So what?
One way is to use the socket module under Gevent, which we can import through the "from gevent import socket". But the more common approach is to use monkey pudding (Monkey patching):
fromGeventImportmonkey; Monkey.patch_socket ()ImportgeventImportSocket URLs= ['www.baidu.com','www.gevent.org','www.python.org']jobs= [Gevent.spawn (socket.gethostbyname, URL) forUrlinchUrls]gevent.joinall (jobs, timeout=5) Print[Job.value forJobinchJobs
The first line of the above code is to the socket standard library Monkey patch, after the socket standard library classes and methods will be replaced with non-blocking, all the other code is not modified, so that the efficiency of the process is really reflected. Other standard libraries in Python also have blocking situations, and Gevent provides a "Monkey.patch_all ()" method to replace all standard libraries.
from import Monkey; Monkey.patch_all ()
The use of monkey patches mixed, but the official website is still recommended to use "Patch_all ()", and in the first line of the program to execute.
Get the process status
The process state has been started and stopped, and can be judged by the "started" property of the co-object and the "Ready ()" method, respectively. For a stopped association, you can use the "successful ()" method to determine whether it runs successfully and does not throw an exception. If the process finishes executing with a return value, it can be obtained through the "value" property. In addition, the exception that occurs during the Greenlet process is not thrown out of the coprocessor, so you need to use the "exception" property of the Coprocessor object to get the exception in the thread. The following example illustrates the use of various methods and properties.
#Coding:utf8Importgeventdefwin ():return 'You win!' deffail ():RaiseException ('You failed!') Winner=Gevent.spawn (Win) loser=gevent.spawn (fail)Printwinner.started#TruePrintloser.started#True #The exception that occurs in Greenlet is not thrown out of the greenlet. #the console will play StackTrace, but the program will not stopTry: Gevent.joinall ([winner, loser])exceptException as E:#This section will never be executed . Print 'This would never be reached' PrintWinner.ready ()#TruePrintLoser.ready ()#True PrintWinner.value#' You win! 'PrintLoser.value#None PrintWinner.successful ()#TruePrintLoser.successful ()#False #This can be done by raise loser.exception or Loser.get ()#to throw an exception in the associationPrintLoser.exception
Run timeout for the co-process
You can pass the timeout parameter in the "Gevent.joinall ()" method to set the timeout, and we can set the time-out in the global scope:
Import gevent from Import = Timeout (2) # 2 secondsdef Wait (): gevent.sleep ( Tentry: gevent.spawn (Wait). Join ()except Timeout: Print('Couldnot complete')
In the example above, we set the time-out to 2 seconds, after which all the threads run, and a "timeout" exception is thrown if more than two seconds. We can also set the timeout within the WITH statement so that the setting is valid only in the WITH statement block:
With Timeout (1): gevent.sleep (10)
In addition, we can specify the exception thrown by the timeout to replace the default "timeout" exception. For example, the timeout in the following example throws our custom "Toolong" exception.
class Toolong (Exception): Pass With Timeout (1, Toolong): gevent.sleep (10)
Communication between co-processes
An event object can be used for asynchronous communication between Greenlet. The "Wait ()" method of the object can block the current coprocessor, and the "set ()" Method can wake up the previously blocked threads. In the example below, 5 waiter will wait for the event evt, and when the setter Association sets the evt event after 3 seconds, all waiter threads are awakened.
#Coding:utf8Importgevent fromGevent.eventImportEvent evt=Event ()defsetter ():Print 'Wait for me'Gevent.sleep (3)#3 seconds to wake up all the threads waiting on the evt Print "Ok, I ' m done"Evt.set ()#Wake up defWaiter ():Print "I ' ll wait for you"evt.wait ()#wait Print 'Finish Waiting'Gevent.joinall ([Gevent.spawn (Setter), Gevent.spawn (Waiter), Gevent.spawn (Waiter), Gevent.spawn (waiter), Gevent.spawn (Waiter), Gevent.spawn (waiter)])
In addition to the event events, Gevent also provides a asyncresult event that can pass messages on wakeup. Let's make the following changes to the setter and waiter in the example above:
fromGevent.eventImportasyncresultaevt=AsyncResult ()defsetter ():Print 'Wait for me'Gevent.sleep (3)#3 seconds to wake up all the threads waiting on the evt Print "Ok, I ' m done"Aevt.set ('hello!')#wake up and deliver the message defWaiter ():Print("I ' ll wait for you") Message= Aevt.get ()#wait, and get the message when it wakes up Print 'Got Wake up message:%s'% message
Queues Queue
The Gevent queue object allows secure access between the Greenlet threads. Run the following program and you will see that 3 consumers will consume the product in the queue separately, and the consumed product will not be picked up by another consumer:
Importgevent fromGevent.queueImportQueue Products=Queue ()defConsumer (name): while notproducts.empty ():Print '%s got product%s'%(name, Products.get ()) gevent.sleep (0)Print '%s Quit' defproducer (): forIinchXrange (1, 10): Products.put (i) Gevent.joinall ([Gevent.spawn (producer), Gevent.spawn (consumer,'Steve'), Gevent.spawn (consumer,'John'), Gevent.spawn (consumer,'Nancy'),])
Both the put and get methods are blocking, both of which have non-blocking versions: Put_nowait and get_nowait. If the queue is empty when the Get method is called, an "Gevent.queue.Empty" exception is thrown. ‘
Signal Volume
The semaphore can be used to limit the number of concurrent threads. It has two methods, acquire and release. As the name implies, acquire is the acquisition of the semaphore, and release is released. When all semaphores have been acquired, the remaining coprocessor can only wait for any one of the co-processes to release the semaphore before it can be run:
Importgevent fromGevent.corosImportboundedsemaphore sem= Boundedsemaphore (2) defworker (N): Sem.acquire ()Print('Worker%i acquired Semaphore'%N) gevent.sleep (0) sem.release ()Print('Worker%i released Semaphore'%N) Gevent.joinall ([Gevent.spawn (Worker, I) forIinchXrange (0, 6)])
In the example above, we initialized the "Boundedsemaphore" semaphore and set the number to 2. So at the same time, only two worker threads can be dispatched. After the program runs, the results are as follows:
1123234455 released Semaphore
Co-local variables
Similar to threads, the co-process also has local variables, which are variables that are accessible only within the current process:
Importgevent fromGevent.localImportLocal Data=Local ()defF1 (): data.x= 1Printdata.xdefF2 ():Try: Printdata.xexceptAttributeerror:Print 'x is not visible'Gevent.joinall ([Gevent.spawn (F1), Gevent.spawn (F2)])
By storing the variable in the local object, you can limit its scope within the current coprocessor and throw an exception when other threads are accessing the variable. The local variables with the same name can be found between the different processes, and do not affect each other. Because the implementation of the co-local variable, it is stored in the "greenlet.getcurrent ()" returned as the key value in the private namespace.
Practical application
Based on flask chat room
https://github.com/sdiehl/minichat/blob/master/app.py
Python network library gevent based on the association process