Fluent Python Learning notes Chapter 18th: processing concurrency with Asyncio packets (i)

The first is the comparison of threads to the association. In this paper, the author uses thread implementation and Asynchio Package Implementation to compare the difference between the two by an example. In a multithreaded sample, the join method is used , and the following join methods are used.

Knowledge Point One:
When a process is started, a primary thread is generated by default because the thread is the smallest unit of the program execution flow, and when multi-threading is set, the main thread creates multiple child threads, in python , by default (in fact , Setdaemon ( False), when the main thread finishes its task, it exits, and the child threads continue to perform their tasks until their task is finished, as shown in the following example:.

def run ():

Time.sleep (2)

Print (' current thread name is :%s\n '% threading.currentthread (). Name)

Time.sleep (2)

If __name__== "__main__":

Start_time=time.time ()

Print (' This is the main thread :%s '% Threading.current_thread (). Name)

Thread_list=[]

For I in range (5):

T=threading. Thread (Target=run)

Thread_list.append (t)

For T in Thread_list:

T.start ()

Print (' main thread end :%s '% Threading.current_thread (). Name)

Print (' total time:%f '% float (time.time ()-start_time))

Operation Result:

Our timing is for the main thread of the timing, the main thread end, the timing also ended, printing out the main thread of time

After the main thread's task is complete, the main thread ends, and the child threads continue to perform their own tasks until all of the child threads have completed their tasks and the program ends.

/usr/bin/python2.7/home/zhf/py_prj/function_test/asy_try.py

This is the main thread : Mainthread

Main thread End : Mainthread

Total elapsed time : 0.000893

The current thread name is : Thread-4

The current thread name is : Thread-1

The current thread name is : Thread-2

The current thread name is : Thread-3

The current thread name is : Thread-5

Knowledge Point two:
When we use the Setdaemon (True) method to set the child thread as the daemon thread, once the main thread is executed, all the threads are terminated, and it is possible that the sub-thread's task is not fully executed and is forced to stop, an example As follows. Set T.setdaemon (True)

For T in Thread_list:

T.setdaemon (True)

T.start ()

The result is as follows: When the main thread ends, all threads end

/usr/bin/python2.7/home/zhf/py_prj/function_test/asy_try.py

This is the main thread : Mainthread

Main thread End : Mainthread

Total elapsed time : 0.000569

Knowledge Point Three:
At this point the role of join is highlighted,the work done byjoin is thread synchronization, that is, after the main thread task ends, into the blocking state, waiting for the execution of other sub-threads to end, the main thread is terminated, Add the following code.

For T in Thread_list:

T.join ()

Run Result: The thread friend exits after the threads run out.

/usr/bin/python2.7/home/zhf/py_prj/function_test/asy_try.py

This is the main thread : Mainthread

The current thread name is : Thread-1

The current thread name is : Thread-2

The current thread name is : Thread-4

The current thread name is : Thread-3

The current thread name is : Thread-5

Main thread End : Mainthread

Total elapsed time : 4.008916

Describes the usage of threads. Now look at the examples in the book using multithreading. This example mainly produces a rotating pointer composed of a |\-/. The code is as follows:

Class Signal:

Go=true

DEF spin (msg,signal):

Write,flush=sys.stdout.write,sys.stdout.flush

For char in itertools.cycle (' |/-\\ '):

status=char+ ' +msg

Write (status)

Flush ()

Write (' \x08 ' *len (status))

Time.sleep (1)

If not signal.go:

Break

Write (' *len (status) + ' \0x8 ' *len (status))

Def slow_function ():

Time.sleep (3)

Return 42

DEF supervisor ():

Result=0

Signal=signal ()

Spinner=threading. Thread (target=spin,args= (' thinking! ', signal))

Print (' Spinner object: ', spinner)

Spinner.start ()

Result=slow_function ()

Signal.go=false

Spinner.join ()

return result

def main ():

Result=supervisor ()

Print (' Answer: ', result)

If __name__== "__main__":

Main ()

The results of the operation are as follows: In this example, the thread is not closed directly, but it is signal.go by the way it exits.

The next introduction to Asyncio. Asyncio 's programming model is a message loop. We get a eventloop Reference directly from the Asyncio module , and then throw the required execution into eventloop to Execute , the asynchronous is implemented . Take a look at some basic concepts:

• Event_loop Event Loop: The program opens an infinite loop, and the programmer registers some functions with the event loop. When the event occurs, the corresponding function is called.
• Coroutine : A Process object, which is a function defined using the Async keyword, whose call does not immediately execute the function, but instead returns a Coprocessor object. The Coprocessor object needs to be registered to the event loop, which is called by the event loop.
• Task tasks: A co-process object is a function that can be suspended natively, and the task is to further encapsulate the process, which contains the various states of the task.
• Future: Represents the outcome of a task that will be executed or not executed in the coming day. There is no essential difference between it and the task.
• async/await keyword :python3.5 is used to define the keywords for the association,Async defines a co-process,await Asynchronous call interface for suspending blocking

We don't read books with Asychio to transform the above example. First Look at how Asyncio works.

The code is as follows

@asyncio. coroutine

def hello ():

Print (' Hello World ')

R=yield from Asyncio.sleep (1)

Print (' Hello again ')

The code can also be written in the following form: Two different ways of writing.

Async def hello ():

Print (' Hello World ')

Await Asyncio.sleep (1)

Print (' Hello again ')

If __name__== "__main__":

Loop=asyncio.get_event_loop ()

Task=loop.create_task (Hello ())

Print (Datetime.datetime.now ())

Print (Task)

Loop.run_until_complete (Task)

Print (Task)

Print (Datetime.datetime.now ())

Loop.close ()

Operation Result:

/usr/bin/python3.6/home/zhf/py_prj/function_test/asy_try.py

2018-03-23 22:00:59.179893

<task pending Coro=

Hello World

Hello again

<task finished Coro=

2018-03-23 22:01:00.181498

The Process object cannot be run directly, when registering the event loop, is actually run_until_complete Span style= "font-family: Arial" > method wraps a thread into a task ( task ) object. The so-called task object is future . In the loop.create_task (Hello ()) , the task is actually in the pending state. In hello Asyncio.sleep (1) takes one second. The status changes to done when the task finishes executing.

Asyncio.ensure_future (coroutine) and loop.create_task (coroutine) can both create a task , Run_until_complete The parameter is a futrue object. When a process is passed in, its interior is automatically encapsulated as a task, andthetask is a subclass of the future

By the above example, we can summarize the followingAsyncioapplication Scenarios. For example, you have a script to3request data from a different server. Sometimes, who knows what the reason, a request sent to one of the servers may have been executed unexpectedly for a long time. Imagine getting data from a second server.Tenseconds. When you wait, the whole script actually does nothing. If you can write a script instead of waiting for a second request but simply skip it, then start executing the third request and then go back to the second request, where you left before executing theThis switching task minimizes idling time. This is the main scenario for Asyncio. This is not the same as the multithreading that is to be preceded. Multithreading is actually multiple tasks parallel processing. And inAsyncioIn fact there is only one time axis. However, it is not a sequential trigger for performing tasks on this timeline. This trigger method is similar to the interrupt. When a task is performedA, an interrupt is generated to perform the taskB,Bwhen the task is finished, return toAcontinue execution of the remaining code at the interrupt point. In the Codeyield fromYou can think of it as an interruption. Therefore, the asynchronous code is executed in one thread. The entire process can be used to represent:

From the following:

1. message loops are executed in the thread

2. getting a task from the queue

3. Each task performs the next action in the process

4. If another coprocessor (await <coroutine_name>)is called in one of the threads, thecontext switch is triggered, the current thread is suspended, and the field environment is saved ( variable, State ), and then load the called Coprocessor

5. if the execution of the process to the blocking part ( blocking I/O,Sleep), the current process hangs, and returns control to the thread's message loop, and then the message loop continues to execute the next task from the queue ... etc.

6. after all the tasks in the queue have been executed, the message loop returns to the first task

Next, let's look at howAsyncio is blocking tasks when multiple tasks are running.

@asyncio. coroutine

def do_work (x):

Print (' Doing work ', X)

Yield from Asyncio.sleep (1)

Return ' done after {}s '. Format (x)

If __name__== "__main__":

Start=time.time ()

Loop=asyncio.get_event_loop ()

Tasks=[asyncio.ensure_future (Do_work (1)), Asyncio.ensure_future (Do_work (2)), Asyncio.ensure_future (Do_work (3))]

Tasks1=[do_work (1), Do_work (2), Do_work (3)]

Loop.run_until_complete (asyncio.wait (Tasks))

Loop.close ()

End=time.time ()

Print ("Total time:{}". Format (End-start))

The results are as follows: There are three tasks in the Code do_work (1), Do_work (2), Do_work (3). when in do_work executed in asyncio.sleep (1) Will give the selection to the main loop, the main loop will go to the queue to find the next task to continue execution.

/usr/bin/python3.6/home/zhf/py_prj/function_test/asy_try.py

Doing work 1

Doing work 2

Doing work 3

Total time:1.0022118091583252

Can be seen in the way of using Asyncio . The total operating mode is 1 seconds. If executed in a sequential manner, the execution time will be 1+2+3=6 seconds. It can be seen that the Asyncio way can greatly shorten the time. Note that you must use asyncio.sleep (1) instead of time.sleep (1), using time.sleep () the way will hang up and do nothing.

We use multithreading to make a comparison:

def run ():

Tasks=[]

Start=time.time ()

T1=threading. Thread (target=do_work_thread,args= (1,))

T2 = Threading. Thread (Target=do_work_thread, args= (2,))

T3 = Threading. Thread (Target=do_work_thread, args= (3,))

Tasks.append (T1)

Tasks.append (T2)

Tasks.append (T3)

For T in tasks:

T.start ()

For T in tasks:

T.join ()

End=time.time ()

Print ("The time is: {}s". Format (End-start))

Running results: You can see that the Asyncio run time and multithreading are the same.

/usr/bin/python3.6/home/zhf/py_prj/function_test/asy_try.py

Doing work 1

Doing work 2

Doing work 3

done after 1s

Done after 3s

Done after 2s

The time is:1.0017800331115723s

Fluent Python Learning notes Chapter 18th: processing concurrency with Asyncio packets (i)