40.python full stack path: process and Thread

Processes and Threads1. What is a process and what is a thread?

process, which is the basic unit of allocating and managing resources in the course of execution, is a dynamic concept, which is the basic unit of computer system resources. Each process has its own address space, either process space or (virtual space). The size of the process space is only related to the number of processor bits, a 16-bit processor has a process space of 216, while the 32-bit processor has a process space of 232. There are at least 5 basic states of the process, which are: initial state, execution state, wait state, ready state, termination state.

threads, in a network or multiuser environment, a server typically needs to receive a large number of concurrent requests from an indeterminate number of users, and creating a process for each request is obviously not a viable , whether in terms of system resource overhead or the efficiency of responding to user requests. Therefore, the concept of the operating system threads is introduced. threads, which are part of a process, can be considered single-threaded by a process without threads. Threads are sometimes referred to as lightweight processes or lightweight processes, and are also a basic unit of CPU scheduling.

2. What is the difference between a process and a thread?

　　　The main difference between processes and threads is that they are different ways to manage operating system resources. The process has a separate address space, and after a process crashes, in protected mode it does not

Other processes have an impact, and threads are just different execution paths in a process. Threads have their own stacks and local variables, but there is no separate address space between the threads, and a

A thread dead is equal to the entire process to die, so the multi-process program is more robust than multi-threaded programs, but in the process of switching, the cost of large resources, efficiency is worse. But

For some concurrent operations that require simultaneous and shared variables, only threads can be used, and processes cannot be used.

3. Global interpreter Lock (GIL)

Add a lock on the process, in order to lock multiple threads, as long as one of the multiple threads to access the CPU, so long lock the resources of the process, do not allow other threads to access the same resource

4. Using multithreading and its common methods

　　Setdaemon (True): The main thread does not wait for a child thread

　　join (N): The main thread waits up to a child thread n seconds

Import threadingimport timedef F1 (A1, A2):    Time.sleep (5)    print (' 666 ') T1 = threading. Thread (TARGET=F1, args= (123, 111)) T1.start () t2 = Threading. Thread (TARGET=F1, args= (123, 111)) T2.start () t3 = Threading. Thread (TARGET=F1, args= (123, 111)) T3.start () "program waits 5 seconds, 3 threads output 666 concurrently"

5. Thread Lock

Import Threadingimport timeglobals_num = 0lock = Threading. Rlock ()  # instantiates a lock def Func ():    lock.acquire ()  # Gets the lock, locks the current access resource    global globals_num    globals_num + = 1    time.sleep (1)    print (globals_num)    lock.release ()  # release lock, release the resource currently accessed for I in range:    t = Threading. Thread (Target=func)    T.start () "Rest a second to output one, because the resource is locked, only wait for the resources to be released to use all the variables can not be the same as the previous example of"

6. Method of the main thread to control the child threads (inter-thread communication)

　　　　Event is one of the most mechanisms of inter-thread communication : One thread sends an Event signal, while the other threads wait for the signal.

Used for the main thread to control the execution of other threads. Events manages a flag that can be set to true using Set () or

Use Clear () to reset to false,wait () for blocking, before flag is true. Flag defaults to False.

　　　　event.wait ([timeout]) : Blocks the thread until the Event object's internal identity bit is set to true or timed out (if parameter timeout is provided).

　　　　Event.set () : Sets the identity bit to ture.

　　　　event.clear () : Sets the identity companion to false.

　　　　Event.isset () : Determines whether the identity bit is ture.

Import threadingdef do:  # loop generates 10 threads    print (' start ')    event.wait ()   # thread is blocked, default is Flase, The wait flag bit is true, and the line friend continues to execute    print (' execute ') Event_obj = threading. Event () for I in range:    t = Threading. Thread (Target=do, args= (Event_obj,))    T.start () event_obj.clear ()   #falge设置为falseinp = input (' input: ') if INP = = ' true ':    event_obj.set ()  # Flage set to True

7. Create the process and use

#创建进程import the main thread of the Multiprocessingimport time# master process creates two sub-processes, and then the thread of the child process executes the F1 method Def F1 (A1):    Time.sleep (2)    print (A1) if __name__ = = ' __main__ ':    t1 = multiprocessing. Process (TARGET=F1, args= (one,))    # T1.daemon = True    T1.start ()  # Run process under Windows needs to be written under __name__ = = ' __main__ '    # t1.join ()    t2 = multiprocessing. Process (TARGET=F1, args= (one,))    # T2.daemon = True    t2.start ()    print (' End ')

8. The difference between a process and a thread on resource sharing issues

　Process:

# process cannot share data, copy original data and use import separately Multiprocessingli = []def foo (i):    li.append (i)    print (' Say hi ', li) if __name__ = = ' __ Main__ ': For    i in range:        p = multiprocessing. Process (Target=foo, args= (i,)) P.start () "Say hi [1]say hi [2]say hi [0]say hi [3]say hi] [4]say        hi [5]say hi [6]say h] I [7]say hi [8]say hi [9]

　　Thread:

#线程与线程之间的内存是共享的import Threadingli = []def foo (i):    li.append (i)    print (' Say hi ', li) if __name__ = = ' __main__ ': For    i in range:        p = Threading. Thread (Target=foo, args= (i,))        P.start () "Say hi [0]say hi [0, 1]say Hi [0, 1, 2]say hi [0, 1, 2, 3]say hi [0, 1, 2, 3, 4]say Hi [0, 1, 2, 3, 4, 5]say hi [0, 1, 2, 3, 4, 5, 6]say hi [0, 1, 2, 3, 4, 5, 6, 7]say hi [0, 1, 2, 3, 4, 5, 6, 7 ]say Hi [0, 1, 2, 3, 4, 5, 6, 7, 8, 9] "

9. Inter-Process memory sharing method (not shared by default)

# manage.dict () shared Data  Special dictionary, support for sharing data between processes from multiprocessing import process, managerdef Foo (i, DIC):    dic[i] = + I    for K, V in Dic.items ():        print (k, v) if __name__ = = ' __main__ ':    manage = Manager ()    dic = Manage.dict () 
   
    # dic = {}    for I in range (2):        p = Process (Target=foo, args= (i, DIC))        P.start ()        p.join ()  # Avoid main process shutdown , the child process cannot connect to the main process, unable to get data    print (DIC) "with manage.dict, data sharing 0 for the  first time only 1 data 0  The second time there are two data yo 1 101{0:100, 1:101} "" In a common dictionary, data does not share 0 1001 101{} "
   

10. Process pool and thread pool

　Process pool: Python provides inside

# python provides process pool from multiprocessing import Poolimport timedef Foo (i):    time.sleep (2)    print (i) if __name__ = = ' __ Main__ ':    pool = Pool (5)  # Max can pack 5 processes, with one plus one, not all in advance    # application process, put back to process pool    # pool.apply (Foo, (1,))    # Pool.apply_async (Func=foo, args= (i,)). Get ()   after the execution, let us know, then put back to the process pool    # After executing the Foo method, assign the return value of the Foo method to the parameter of the bar method    # Bar method, which we call the callback function for    I in range (+):        # pool.apply (Foo, (i)) apply the application at  the time of one application and execute the        pool.apply_ Async (Func=foo, args= (i))  # Concurrent, can also set callback function        print (' 666666666666 ')    pool.close ()    pool.join ()

　　Low version thread pool : The disadvantage is that after a thread executes a task, it cannot be reused and waits for Python to destroy it

Import Queueimport threadingimport timeclass ThreadPool (object):    def __init__ (self, max_num=20):        self.queue = Queue. Queue (Max_num)  # Creates a queue with a maximum length of 20 for        I in range (max_num):            self.queue.put (Threading. Thread)  # Pass the class name, queue with 20 class names, all pointing to the same class    def get_thread (self):        return Self.queue.get ()  # When the queue is empty, get waits    def add_thread (self):        self.queue.put (Threading. Thread) def func (pool, A1):    Time.sleep (1)    print (A1)    pool.add_thread () p = ThreadPool (Ten) for I in range ( :    thread = P.get_thread ()  # RET is equivalent to threading. Thread, which is a class name    t = Thread (Target=func, args= (P, i))  # Creates a thread that passes the thread pool object into the Func, and then puts the thread back into the thread pool    at the end of the Func T.start ()  # start Thread    # put the class in the queue, the thread is waiting to be destroyed, it is unreasonable that we can optimize, the thread continues to do the second thing    # put the task in the queue, let the thread keep fetching the task, reuse the thread

　　

40.python full stack path: process and Thread