Python Full Stack development * Threads Queue Thread Pool association * 180731

One. Thread queue

Queue:
1.Queue
    Advanced First Out
    Self-locking  data security
    From queue import queue
    From multiprocessing import queue    (IPC queue)
2.LifoQueue LIFO
    Last in, first out
    Self-locking  data security

 fromQueueImportLifoqueue LQ=lifoqueue (5) Lq.put (123) Lq.put (666) Lq.put (888) Lq.put (999) Lq.put (" Love")    Print(Lq.put_nowait ("Miss"))#error queue. full    Print(LQ)#<queue. Lifoqueue object at 0x0000017901bc8c88>    Print(Lq.get ())# Love    Print(Lq.get ())#999    Print(Lq.get ())#888    Print(Lq.get ())#666    Print(Lq.get ())#123    #print (lq.get_nowait ()) #报错 queue. Empty

3.PriorityQueue Priority Queue
    (Put the tuple, the numbers from small to large, the English alphabet in order of ASCII code)

 fromQueueImportPriorityqueue PQ=priorityqueue (4) Pq.put (10,"AAA")) Pq.put (5,"S")) Pq.put (5,"CCC")) Pq.put (10,"zzz"))    #pq.put_nowait ((Ten, "BBB")) #报错queue. full    Print(PQ)#<queue. Priorityqueue object at 0x000001d6fef38c50> print (Pq.get ())    Print(Pq.get ())#(5, ' CCC ')    Print(Pq.get ())#(Ten, ' AAA ')    Print(Pq.get ())#(the ' zzz ')    Print(Pq.get ())#(the ' BBB ')    #print (Pq.get_nowait ()) # error queue. Empty

Second-tier pool

Multiprocessing module comes with process pool
Threading module does not have pool (no thread pool)
Concurrent.futures helps you manage the thread pool and process pool
    Highly encapsulated
    Uniform and uniform usage of the process pool/thread pool

Import Time fromThreadingImportCurrentThread fromConcurrent.futuresImportProcesspoolexecutor fromConcurrent.futuresImportThreadpoolexecutordeffunc (i): Time.sleep (1)        Print("In %s%s"%(I,currentthread ()))returnI**2defBack (FN):Print(Fn.result (), CurrentThread ()) T=threadpoolexecutor (5) ret_l=[]     forIinchRange (20): Ret=T.submit (func,i). Add_done_callback (back)#ret_l.append (ret)T.shutdown (Wait=true)#can be omitted in parentheses    #For ret in ret_l:    #print (Ret.result ())    Print(666)

Related Methods of Threadpoolexecutor:
    1.t.map method to   start a multithreaded task    # T.map (Func,range ())  Override for  submit
    2.t.submit (Func,*args,**kwargs)  Asynchronous Submit task
    3.t.shutdown (wait=true)  is equivalent to the Pool.close () +pool.join () Operation  Synchronization control of the process pool
        Wait=true, wait until all the tasks in the pool have been completed and the resources have been reclaimed before continuing
        Wait=false, returns immediately, and does not wait for the task in the pool to complete
        Submit and map must be before shutdown
    4.result Getting Results    ret.result ()
    5. Callback function Add_done_callback (back)
        The parameter that is received inside the callback function is an object that needs to get the return value through result
        Executing in the main process
Three. Co-process

Process: The smallest unit of resource allocation
Threads: The smallest unit of CPU scheduling
Co-process: ability to switch between multiple tasks on a thread basis
    Save on thread-open consumption
    Scheduling from the level of Python code
        A normal thread is the smallest unit of CPU scheduling
        The scheduling of the association is not done by the operating system.
(a) The mechanism of yield is the co-process

def func ():         Print (1)        x=yield"aaa"        Print (x)         yield " BBB "     g=func ()    print(next (g))    print(G.send ( " *** "))

(b). The ability to switch between multiple functions--

def consumer ():          while True:            x=yield            Print (x)     def producer ():        g=consumer ()        next (g)        for in range (  ):            g.send (i)    producer ()

Yeild only switch between programs, no time to re-use any IO operations
Greenlet (third-party module) Program Context Switch
CMD:PIP3  Install module name  third-party modules
(iii). Greenlet
      Simple program switching of the process module time consuming

 Import Time fromGreenletImportGreenletdefeat ():Print('Eat') Time.sleep (1) G2.switch ()Print("Finished eating") Time.sleep (1) G2.switch ()defPlay ():Print("Play") Time.sleep (1) G1.switch ()Print("playing beauty.") G1=Greenlet (Eat) G2=Greenlet (play) G1.switch ()

(d). gevent
    Switch  to use the IO to reduce the time consumed by the IO operation
    Greenlet is the bottom of gevent.
    Gevent is based on the Greenlet implementation.
    Python code switching in the control program
First edition:

Import TimeImportgevent fromGeventImportMonkeydefeat ():Print("Eat") Gevent.sleep (2)        Print("Finished eating")    defPlay ():Print("Play") Gevent.sleep (2)        Print("playing beauty.") G1=Gevent.spawn (Eat) G2=Gevent.spawn (play) G1.join ()#waiting for G1 to endG2.join ()#waiting for G2 to end

Second Edition
To use gevent, you need to place the from Gevent import Monkey;monkey.patch_all () to the beginning of the file

 fromGeventImportMonkey;monkey.patch_all ()Import TimeImportgeventdefEat (name):Print("Eat") Time.sleep (2)        Print("%s finished eating"%name)defPlay ():Print("Play") Time.sleep (2)        Print("playing beauty.") G1=gevent.spawn (Eat,"Alex")#The first parameter in parentheses is the function name, which can be either a positional parameter or a keyword argument, which is passed to eat.G2=Gevent.spawn (play) Gevent.joinall ([g1,g2])#G1.join () and g2.join () merge into one.    Print(G1.value)#None

Four. Co-socket (TCP)
Server code

 fromGeventImportMonkey;monkey.patch_all ()ImportSocketImportgeventdefTalk (conn): whileTrue:conn.send (b'Hallo')        Print(CONN.RECV (1024)) SK=socket.socket () Sk.bind ("127.0.0.1", 9902) ) Sk.listen () whiletrue:conn,addr=sk.accept () gevent.spawn (talk,conn)

Client code

ImportSocket fromThreadingImportThreaddefclient (): SK=socket.socket () Sk.connect ("127.0.0.1", 9902))     whileTrue:Print(SK.RECV (1024)) Sk.send (b'Hi') forIinchRange (5): Thread (Target=client). Start ()

Python Full Stack development * Threads Queue Thread Pool association * 180731