Process, thread, coroutine, and Python in python

Process, thread, coroutine, and Python in python

The relationships and differences between processes, threads, and coroutines have also plagued me for a while. I have some experiences recently.

A process has its own independent heap and stack. It neither shares the heap nor shares the stack. The process is scheduled by the operating system.

The thread has its own independent stack and shared heap. The shared heap does not share the stack. The thread is also scheduled by the operating system (Standard thread is true ).

The coroutine shares the heap like the thread and does not share the stack. The cooutine shows scheduling in the coroutine code.

The difference between the Process and the other two is obvious.

The difference between coroutine and thread is that coroutine avoids meaningless scheduling, which can improve performance. However, programmers must take the scheduling responsibilities themselves, coroutine also loses the ability to use multiple CPUs for standard threads.

Python thread

Definition: Threading is used to provide thread-related operations. A thread is the smallest unit of work in an application.

#!/usr/bin/env python# -*- coding:utf-8 -*-import threadingimport timedef show(arg):time.sleep(1)print 'thread'+str(arg)for i in range(10):t = threading.Thread(target=show, args=(i,))t.start()print 'main thread stop 

The above Code creates 10 "foreground" threads, and then the controller is handed over to the CPU, the CPU Schedules according to the specified algorithm, and executes commands in parts.

More methods:

• The start thread is ready for CPU scheduling.

• SetName indicates the thread name.

• GetName: Get the thread name

• SetDaemon is set to the background thread or foreground thread (default)

If it is a background thread, the background thread is also running during the main thread execution process. After the main thread is executed, the background thread stops no matter whether it is successful or not.

If it is a foreground thread, the foreground thread is also running during the main thread execution. After the main thread is executed, the program stops after the foreground thread is executed.

• Join executes each thread one by one, and continues to run after execution. This method makes multithreading meaningless.

• After the run thread is scheduled by the cpu, the run method of the thread object is automatically executed.

Thread lock

Since threads are randomly scheduled, and each thread may only execute n executions, the CPU then executes other threads. Therefore, the following problems may occur:

import threadingimport timegl_num = 0def show(arg):global gl_numtime.sleep(1)gl_num +=1print gl_numfor i in range(10):t = threading.Thread(target=show, args=(i,))t.start()print 'main thread stop' import threadingimport timegl_num = 0lock = threading.RLock()def Func():lock.acquire()global gl_numgl_num +=1time.sleep(1)print gl_numlock.release()for i in range(10):t = threading.Thread(target=Func)t.start() 

Event

The events of the python thread are used by the main thread to control the execution of other threads. The events mainly provide three methods: set, wait, and clear.

Event processing mechanism: a global "Flag" is defined. If the "Flag" value is False, when the program executes the event. the wait method is blocked. If the "Flag" value is True, the event. the wait method is no longer blocked.

• Clear: Set "Flag" to False

• Set: set "Flag" to True

#!/usr/bin/env python# -*- coding:utf-8 -*-import threadingdef do(event):print 'start'event.wait()print 'execute'event_obj = threading.Event()for i in range(10):t = threading.Thread(target=do, args=(event_obj,))t.start()event_obj.clear()inp = raw_input('input:')if inp == 'true':event_obj.set() 

Python Process

from multiprocessing import Processimport threadingimport timedef foo(i):print 'say hi',ifor i in range(10):p = Process(target=foo,args=(i,))p.start() 

Note: because the data between processes must be one copy of each other, it is very costly to create a process.

Process data sharing

Each process holds one copy of data, and data cannot be shared by default.

#!/usr/bin/env python#coding:utf-8from multiprocessing import Processfrom multiprocessing import Managerimport timeli = []def foo(i):li.append(i)print 'say hi',lifor i in range(10):p = Process(target=foo,args=(i,))p.start()print ('ending',li) 

# Method 1, Array

from multiprocessing import Process,Arraytemp = Array('i', [11,22,33,44])def Foo(i):temp[i] = 100+ifor item in temp:print i,'----->',itemfor i in range(2):p = Process(target=Foo,args=(i,))p.start()

# Method 2: manage. dict () share data

from multiprocessing import Process,Managermanage = Manager()dic = manage.dict()def Foo(i):dic[i] = 100+iprint dic.values()for i in range(2):p = Process(target=Foo,args=(i,))p.start()p.join() 'c': ctypes.c_char, 'u': ctypes.c_wchar,'b': ctypes.c_byte, 'B': ctypes.c_ubyte,'h': ctypes.c_short, 'H': ctypes.c_ushort,'i': ctypes.c_int, 'I': ctypes.c_uint,'l': ctypes.c_long, 'L': ctypes.c_ulong,'f': ctypes.c_float, 'd': ctypes.c_double 

When a process is created (not used), the shared data is obtained to the sub-process. After the sub-process is executed, the original value is assigned.

#! /Usr/bin/env python #-*-coding: UTF-8-*-from multiprocessing import Process, Array, RLockdef Foo (lock, temp, I ): "add 0th to 100" "lock. acquire () temp [0] = 100 + ifor item in temp: print I, '----->', itemlock. release () lock = RLock () temp = Array ('I', [11, 22, 33, 44]) for I in range (20 ): p = Process (target = Foo, args = (lock, temp, I,) p. start ()

Process pool

A process sequence is maintained in the Process pool. When used, a process is obtained from the process pool. If there is no usable process in the process pool sequence, the program will wait, until there are available processes in the process pool.

There are two methods in the process pool:

• Apply

• Apply_async

#!/usr/bin/env python# -*- coding:utf-8 -*-from multiprocessing import Process,Poolimport timedef Foo(i):time.sleep(2)return i+100def Bar(arg):print argpool = Pool(5)#print pool.apply(Foo,(1,))#print pool.apply_async(func =Foo, args=(1,)).get()for i in range(10):pool.apply_async(func=Foo, args=(i,),callback=Bar)print 'end'pool.close()

Pool. join () # Shut down the processes in the process pool after they are executed. If comments are made, the program will be closed directly.

Coroutine

The operations of threads and processes are system interfaces triggered by programs, and the executors are systems. The operations of coroutines are programmers.

The meaning of coroutine: For multi-threaded applications, the CPU uses slices to switch the execution between threads. It takes time to switch the threads (Save the status and continue next time ). Coroutine, only one thread is used to specify the execution sequence of a code block in one thread.

Application scenarios of coroutine: It is applicable to coroutine when there are a large number of operations without CPU (IO) in the program;

Greenlet

#!/usr/bin/env python# -*- coding:utf-8 -*-from greenlet import greenletdef test1():print 12gr2.switch()print 34gr2.switch()def test2():print 56gr1.switch()print 78gr1 = greenlet(test1)gr2 = greenlet(test2)gr1.switch() 

Gevent

import geventdef foo():print('Running in foo')gevent.sleep(0)print('Explicit context switch to foo again')def bar():print('Explicit context to bar')gevent.sleep(0)print('Implicit context switch back to bar')gevent.joinall([gevent.spawn(foo),gevent.spawn(bar),]) 

Automatic Switch upon IO operation:

from gevent import monkey; monkey.patch_all()import geventimport urllib2def f(url):print('GET: %s' % url)resp = urllib2.urlopen(url)data = resp.read()print('%d bytes received from %s.' % (len(data), url))gevent.joinall([gevent.spawn(f, 'https://www.python.org/'),gevent.spawn(f, 'https://www.yahoo.com/'),gevent.spawn(f, 'https://github.com/'),]) 

The above is a small series of knowledge about processes, threads, and coroutines in Python. I hope to help you!

Articles you may be interested in:

• Differences between threads and processes and Python code instances
• Analysis on the use of multi-process and multi-thread in Python
• Exploring the sharing of variables between threads in Python multi-process Programming
• Python multi-thread, asynchronous, and multi-process crawler implementation code