python-co-process

1. Co-process

Co-process, also known as micro-threading, fiber. English name Coroutine. One sentence explains what a thread is: The process is a lightweight thread that is user-state. The co-process has its own register context and stack. When the schedule is switched, the register context and stack are saved elsewhere, and the previously saved register context and stack are restored when it is cut back. Thus: The process can retain the state of the last invocation (that is, a specific combination of all local states), each time the procedure is re-entered, which is equivalent to the state of the last call, in other words: The position of the logical flow at the last departure.

2. Advantages and disadvantages of the co-process.

Benefits of the co-process: No need for thread context switching overhead without atomic operation locking and synchronization overhead " Atomic Operations (atomic operation) are not required synchronized " , atomic operations are operations that are not interrupted by the thread scheduling mechanism, which, once started, runs until the end without any context switch (switching to another thread).
An atomic operation can be a step or multiple steps, but its order cannot be disrupted, or it can be cut off only the execution part. As a whole is the nucleus of atomicity. Easy switching of control flow, simplified programming model high concurrency + high scalability + low cost: One CPU support for tens of thousands of processes is not a problem. Therefore, it is suitable for high concurrency processing. Disadvantage: Unable to take advantage of multicore resources: The nature of the process is single-threaded, it can not be a single CPU at the same time multiple cores, the process needs and processes to run on multi-CPU. Of course, most of the applications we write on a daily basis are not necessary, except for CPU-intensive applications. Blocking (Blocking) operations (such as IO) can block the entire program

The 3.python greenlet module.

Greenlet is a C implementation of the process module, compared with the yield of Python, it can allow you to arbitrarily switch between any function, without the need to declare the function as generato manual switch.

A simple example of 3.1 greenlet.

def T1 ():     Print (one)    gr2.switch ()    print(    gr2.switch ()def  T2 ():     Print(    gr1.switch    )print(==  Greenlet (T2) Gr1.switch ()

The 4.python gevent module.

Gevent is a third-party library that makes it easy to implement concurrent or asynchronous programming through Gevent, and the main pattern used in Gevent is Greenlet, which is a lightweight coprocessor that accesses Python in the form of a C extension module. Greenlet all run inside the main program operating system process, but they are dispatched in a collaborative manner.

A simple example of 4.1 gevent

ImportGevent,timedefT1 ():Print(11) Gevent.sleep (1)    Print(33)defT2 ():Print(22) Gevent.sleep (3)    Print(44) Start_time=time.time () Gevent.joinall ([Gevent.spawn (T1), Gevent.spawn (T2),])Print('cost:{0}'. Format (Time.time ()-start_time))

5. The process implements a simple crawler and contrasts, serial and parallel execution times.

Importgevent,time,requests fromGeventImportMonkey"""Mark All IO operations on the current program. Let Gevent know the end time of the IO operation in the program. """Monkey.patch_all ()defget1 (URL):Print('Get URL: {0}'. Format (URL)) Rep=requests.get (URL) rep.encoding='Utf-8'    Print(rep) start_time=time.time () url_lists= [    'http://www.gec-edu.org/',    'http://www.baidu.com/',    'http://www.sina.com.cn/',] forIteminchUrl_lists:get1 (item)#Gevent.joinall ()Print('cost:{0}'. Format (Time.time ()-start_time)) Aync_start_time=time.time () Gevent.joinall ([Gevent.spawn (Get1 (Url_lists[0])), Gevent.spawn (Get1 (url_lists[1]), Gevent.spawn (Get1 (url_lists[2])),])Print('Aync cost:{0}'. Format (Time.time ()-aync_start_time))

6. Using the process to achieve socket multi-concurrency.

6.1 client.py

ImportGevent,socket,timedefrun (): Client ()defClient (): Client=socket.socket () Client.connect ('127.0.0.1', 8089))     whileTrue:input_value= Input (">>:"). Strip ()if  notInput_value:Continueclient.send (Input_value.encode ('Utf-8')) Rev_data= Client.recv (1024x768). Decode ('Utf-8')        Print(Rev_data)Continueif __name__=='__main__': Run ()

6.2 servier.py

ImportGevent,socket,time fromGeventImportMonkey"""the multi-concurrency of Socket server is realized by using Gevent. IDEA: Each request switches one process processing"""Monkey.patch_all ()defserver (): Server=Socket.socket ()#Binding Listening PortsServer.bind (('127.0.0.1', 8089))    #MonitorServer.listen () whiletrue:connect,address=server.accept ()Print('Server Listen request ...', address)"""If a request comes in, it is referred to a co-process"""gevent.spawn (handle_request,connect,address)defhandle_request (conncet,address):Try:         whileTrue:rev_data= Conncet.recv (1024x768). Decode ('Utf-8')            if  notRev_data: Break            Print(Address,'Server recv: {0}'. Format (rev_data)) Conncet.send ('recv ok! Request data: {0}'. Format (Rev_data). Encode ('Utf-8'))    exceptException as E:Print(e)finally: Conncet.close ()defrun (): Server ()if __name__=='__main__': Run ()

python-co-process