threads, processes, and co-routines

Python Threads

Threading is used to provide thread-related operations, which are the smallest unit of work in an application.

#!/usr/bin/env python#-*-coding:utf-8-*-import threadingimport time def Show (ARG): Time.sleep (1) print ' thread ' + STR (ARG) for I in range: T = Threading. Thread (target=show, args= (i,)) T.start () print ' main thread stop '

　　

The code above creates 10 "foreground" threads, then the controller is handed over to the CPU,CPU according to the specified algorithm for scheduling, shard execution instructions.

More ways:

• Start thread is ready to wait for CPU scheduling
• SetName setting a name for a thread
• GetName Get thread Name
• Setdaemon set to background thread or foreground thread (default)
  If it is a background thread, during the main thread execution, the background thread is also in progress, and after the main thread finishes executing, the background thread stops regardless of success or not.
  If it is the foreground thread, during the main thread execution, the foreground thread is also in progress, and after the main thread finishes executing, wait for the foreground thread to finish, the program stops
• The join executes each thread one by one and continues execution after execution, making multithreading meaningless
• The Run method that executes the thread object automatically after the run thread is dispatched by the CPU

Thread Lock

The CPU then executes other threads because the threads are randomly dispatched, and each thread may execute only n execution. Therefore, the following problems may occur:

#!/usr/bin/env python#-*-coding:utf-8-*-ImportThreadingImportTimegl_num=0defShow (ARG):Globalgl_num Time.sleep (1) Gl_num+=1PrintGl_num forIinchRange (10): T= Threading. Thread (Target=show, args=(i,)) T.start ()Print 'Main thread Stop'Lock not used

Lock not used

#!/usr/bin/env python#coding:utf-8 Import threadingimport Time Gl_num = 0 lock = Threading.       Rlock () def Func (): Lock.acquire () global gl_num gl_num +=1 time.sleep (1) Print Gl_num lock.release () For I in range: T = Threading. Thread (Target=func) T.start ()

　　

Event

The events of the Python thread are used by the main thread to control the execution of other threads, and the event provides three methods set, wait, clear.

Event handling mechanism: A global definition of a "flag", if the "flag" value is False, then when the program executes the Event.wait method is blocked, if the "flag" value is true, then the Event.wait method will no longer block.

• Clear: Set "Flag" to False
• Set: Sets "Flag" to True

#!/usr/bin/env python#-*-coding:utf-8-*-import Threading def Do (event): print ' Start ' event.wait () print ' ex Ecute ' Event_obj = threading. Event () for I in range: T = Threading. Thread (Target=do, args= (Event_obj,)) T.start () event_obj.clear () InP = raw_input (' input: ') if InP = = ' true ': event_obj . Set ()

Python Process

From multiprocessing import processimport threadingimport time def foo (i): print ' Say hi ', I-I in range: p = Process (target=foo,args= (i,)) P.start ()

　　

Note: Because the data between processes needs to be held separately, the creation process requires very large overhead.

Process data sharing

The process holds one piece of data, and the data is not shared by default

#!/usr/bin/env python#Coding:utf-8  fromMultiprocessingImportProcess fromMultiprocessingImportManagerImportTime Li= [] deffoo (i): Li.append (i)Print 'say hi', Li forIinchRange (10): P= Process (target=foo,args=(i,)) P.start ()Print 'ending', there is no data sharing between Li processes by default

no data sharing between processes by default

#方法一, arrayfrom multiprocessing Import process,arraytemp = Array (' i ', [11,22,33,44]) def Foo (i): temp[i] = 100+i for Item in Temp:print I, '-----> ', item for I in range (2): P = Process (target=foo,args= (i,)) P.start () #方法二: MA Nage.dict () shared data from multiprocessing import Process,manager manage = Manager () dic = Manage.dict () def Foo (i): dic[i] = 1 00+i print dic.values () for I in Range (2): P = Process (target=foo,args= (i,)) P.start () P.join ()

'C': Ctypes.c_char,'u': Ctypes.c_wchar,'b': Ctypes.c_byte,'B': Ctypes.c_ubyte,'h': Ctypes.c_short,'H': Ctypes.c_ushort,'I': Ctypes.c_int,'I': Ctypes.c_uint,'L': Ctypes.c_long,'L': Ctypes.c_ulong,'F': Ctypes.c_float,'D': ctypes.c_double type Correspondence Table

Type Correspondence Table

When the process is created (when not in use), the shared data is taken to the child process, and is then assigned to the original value when the process finishes executing.

#!/usr/bin/env python#-*-coding:utf-8-*- fromMultiprocessingImportProcess, Array, RlockdefFoo (lock,temp,i):"""Add a No. 0 number to the"""Lock.acquire () temp[0]= 100+I forIteminchTemp:PrintI'----->', item lock.release () lock=rlock () temp= Array ('I', [11, 22, 33, 44]) forIinchRange (20): P= Process (target=foo,args=(Lock,temp,i,)) P.start () process Lock instance

Process Lock Instance

Process Pool

A process sequence is maintained internally by the process pool, and when used, a process is fetched in the process pool, and the program waits until a process is available in the process pool sequence if there are no incoming processes available for use.

There are two methods in a process pool:

• Apply
• Apply_async

#!/usr/bin/env python#-*-coding:utf-8-*-from multiprocessing import process,poolimport time def Foo (i): Time.sleep (2) Return i+100 def Bar (ARG): print arg pool = Pool (5) #print pool.apply (Foo, (1,)) #print Pool.apply_async (func =fo O, args= (1,)). Get () for I in range: Pool.apply_async (Func=foo, args= (i,), callback=bar) print ' End ' pool.close () poo L.join () #进程池中进程执行完毕后再关闭, if the comment, then the program closes directly.

co-process

The operation of the thread and process is triggered by the program to trigger the system interface, the final performer is the system, and the operation of the coprocessor is the programmer.

The significance of the existence of the process: for multi-threaded applications, the CPU by slicing the way to switch between threads of execution, thread switching takes time (save state, next continue). , only one thread is used, and a code block execution order is specified in one thread.

Application scenario: When there are a large number of operations in the program that do not require the CPU (IO), it is suitable for the association process;

Greenlet

#!/usr/bin/env python#-*-coding:utf-8-*-from greenlet import Greenlet def test1 (): Print Gr2.switch () pr int Gr2.switch () def test2 (): Print Gr1.switch () print Gr1 = Greenlet (test1) GR2 = Greenlet (test2) Gr1 . switch ()

Gevent

Import Gevent def foo (): Print (' Running in foo ') gevent.sleep (0) print (' Explicit context switch to foo again ') de F bar (): Print (' Explicit context to bar ') gevent.sleep (0) print (' Implicit context switch back to bar ') Gevent.joi Nall ([Gevent.spawn (foo), Gevent.spawn (bar),])

Automatic switching of IO operation encountered:

 fromGeventImportmonkey; Monkey.patch_all ()ImportgeventImportUrllib2deff (URL):Print('GET:%s'%URL) Resp=urllib2.urlopen (URL) data=Resp.read ()Print('%d bytes received from%s.'%(len (data), URL)) Gevent.joinall ([Gevent.spawn (F,'https://www.python.org/'), Gevent.spawn (F,'https://www.yahoo.com/'), Gevent.spawn (F,'https://github.com/'),])

thread pool:programme One:

#!/usr/bin/env python#-*-coding:utf-8-*-import queueimport threading  class ThreadPool (object):      def __init__ (self, max_num=20):         Self.queue = Queue.queue (max_num)         for i in Xrange (max_num):             self.queue.put (Threading. Thread)      def get_thread (self):         Return Self.queue.get ()      def add_thread (self):         self.queue.put (Threading. Thread)   "" "Pool = ThreadPool ( def) func (ARG, p):     print arg     import Time    time.sleep (2)     p.add_thread ()   for I in Xrange (+):     thread = Pool.get_thread ()     t = Thread (target=Func, args= (i, Pool))     t.start () "" " 

　Scenario Two:

#!/usr/bin/env python#-*-coding:utf-8-*-import queueimport threadingimport contextlibimport timeStopEvent = Object () Class ThreadPool (object): Def __init__ (self, max_num): self.q = queue. Queue () Self.max_num = max_num Self.terminal = False self.generate_list = [] Self.free_list = [] def run (self, func, args, Callback=none): "" "Thread pool performs a task:p Aram Func: Task function:p Aram args: any Functions Required Parameters:p Aram callback: A callback function executed after the failure or success of the task, the callback function has two parameters 1, the task function execution state, 2, the task function return value (default is None, that is: Do not execute the callback function): return: If the thread pool is already Terminates, returns true otherwise none "" "If Len (self.free_list) = = 0 and len (self.generate_list) < self.max_num:s        Elf.generate_thread () W = (func, args, callback,) Self.q.put (W) def Generate_thread (self): "" " Create a thread "" "T = Threading. Thread (Target=self.call) T.start () def call (self): "" Loop to get the task function and execute the task function "" "Current_ thread = Threading.currentThread self.generate_list.append (current_thread) event = Self.q.get () while event! = Stopevent: Func, arguments, callback = Event Try:result = func (*arguments) status = T Rue except Exception as E:status = False result = e If callback is no                    T None:try:callback (status, result) except Exception as E: Pass if self.terminal: # False event = stopevent Else:with sel F.worker_state (self.free_list,current_thread): event = Self.q.get () else:self.genera             Te_list.remove (Current_thread) @contextlib. ContextManager def worker_state (self,x,v): X.append (v) Try: Yield Finally:x.remove (v) def close (self): num = Len (self.generate_list) W Hile Num:selF.q.put (stopevent) num-= 1 # terminating thread (emptying queue) def terminate (self): self.terminal = True while SE Lf.generate_list:self.q.put (stopevent) self.q.empty () Import timedef work (i): Time.sleep (1) Print ( i) Pool = ThreadPool () for item in range: Pool.run (func=work, args= (item)) # Pool.terminate () #立即终止所有线程

Programme III,

From concurrent.futures import threadpoolexecutorimport timedef F1 (a):    time.sleep (2)    print (a)    return 1pool=threadpoolexecutor (5) for I in range:    a=pool.submit (f1,i)    # X=a.result () #获取返回值, if any, will block

threads, processes, and co-routines