Python concurrent programming

Multi-process and multi-threaded daemon differences

First, make it clear that either multi-process or multi-threaded, the master process or the main thread will wait for the child process or the child thread to exit before exiting.
Whether it is a process or a thread, follow: Guardian xxx will wait for the main xxx to be destroyed after the run is complete. It should be emphasized that the operation is not terminated
1. For the main process, running complete means that the main process code is running
2. To the main thread said, run complete refers to the main thread in the process of all non-daemon threads run complete, the main thread is run complete
Other words:

1. The main process is finished after its code is finished (the daemon is recycled at this point), and then the main process waits until the non-daemon child processes have finished running to reclaim the child process's resources (otherwise it will produce a zombie process) before it ends.
   2. The main thread runs after the other non-daemon threads have finished running (the daemon is recycled at this point). Because the end of the main thread means the end of the process, the resources of the process as a whole are recycled, and the process must ensure that the non-daemon threads are finished before they end.

import osimport timefrom multiprocessing import Processdef task1():    while True:        print('task1', os.getpid())        time.sleep(1)def task2():    while True:        print('task2')        time.sleep(1.5)if __name__ == '__main__':    p1 = Process(target=task1)    p1.daemon = True    p1.start()    p2 = Process(target=task2)    p2.start()    # task1 不会被执行，因为进程的开启是比线程慢的，所以一般情况下是主进程代码执行完毕再执行子进程    print('main over')

GIL Lock

The Gil Lock is a mutex that the CPython interpreter helps us add, and this lock is based on the memory management mechanism. If you do not have this lock, then consider that multiple threads can only be processed by one CPU at a time, because a multi-threaded concurrency without locks will certainly be designed to resource preemption. The garbage collection mechanism has not finished the work has been another thread snatched the CPU execution permissions, just this thread and garbage collection to deal with a variable to do related operations (such as add 1), then this situation garbage collection is meaningless. Therefore, a lock is added to the outside of the process space, and if the garbage collection mechanism grabs the lock, the garbage collection can be completed by releasing the lock so that the garbage collection mechanism is realized.

What is the difference between the Gil Lock and our own lock in the program declaration? The Gil Lock can be seen as a lock on the entire process space exit, and our own stated lock is used to lock the data inside the process.

Multi-process Lock

Multi-process is data isolation, why do you need a lock? Because multiple processes are data-isolated, they share file systems and print terminals. If you are opening multiple process team file process read and write operations, then you need to use the lock

from multiprocessing import Process,Lockimport time,jsondef search():    dic=json.load(open('ticket.txt'))    print('\033[43m剩余票数%s\033[0m' %dic['count'])def get():    dic=json.load(open('ticket.txt'))    time.sleep(0.1) # 模拟读数据的网络延迟,这里是为了等待其他进程开启并完成load操作    if dic['count'] > 0:        dic['count'] -= 1        time.sleep(0.2) # 模拟写数据的网络延迟,以防dump的w模式清空文件但是另一个进程在search的时候load空文件会报错        json.dump(dic,open('ticket.txt','w'))        print('\033[43m购票成功\033[0m')def task(lock):    search()    get()if __name__ == '__main__':    lock=Lock()    for i in range(100): #模拟并发100个客户端抢票        p=Process(target=task,args=(lock,))        p.start()

Locking

from multiprocessing import Process,Lockimport time,jsonfrom multiprocessing import Lockdef search():    dic=json.load(open('ticket.txt'))    print('\033[43m剩余票数%s\033[0m' %dic['count'])def get():    dic=json.load(open('ticket.txt'))    time.sleep(0.1) # 模拟读数据的网络延迟,这里是为了等待其他进程开启并完成load操作    if dic['count'] > 0:        dic['count'] -= 1        time.sleep(0.2) # 模拟写数据的网络延迟,以防dump的w模式清空文件但是另一个进程在search的时候load空文件会报错        json.dump(dic,open('ticket.txt','w'))        print('\033[43m购票成功\033[0m')def task(lock):    search()    lock.acquire()    get()    lock.release()if __name__ == '__main__':    lock=Lock()    for i in range(100): #模拟并发100个客户端抢票        p=Process(target=task,args=(lock,))        p.start()

Instead of locking inside the Get function, it encapsulates a task function that locks the get () inside the task function.

Queue

Both the process and thread queues have been locked for us, and the queue can be thrown into a custom object, and can also throw a none object. Threads can share global variables, and while processes cannot share global variables, there can be ways to share data. Share with Thread
Data will have to seize resources, the process if the use of shared data, there will also be a situation of resource competition. The manager of multiprocessing is actually another process that opens up a shared memory in this process.

Pool of multiprocessing

Synchronous usage (not commonly used)

from multiprocessing import Poolimport os,timedef work(n):    print('%s run' %os.getpid())    time.sleep(3)    return n**2if __name__ == '__main__':    p=Pool(3) #进程池中从无到有创建三个进程,以后一直是这三个进程在执行任务    res_l=[]    for i in range(10):        res=p.apply(work,args=(i,)) #同步调用，直到本次任务执行完毕拿到res，等待任务work执行的过程中可能有阻塞也可能没有阻塞，但不管该任务是否存在阻塞，同步调用都会在原地等着，只是等的过程中若是任务发生了阻塞就会被夺走cpu的执行权限        res_l.append(res)    print(res_l)

Asynchronous

from multiprocessing import Poolimport os,timedef work(n):    time.sleep(1)    return n**2if __name__ == '__main__':    p=Pool(os.cpu_count()) #进程池中从无到有创建三个进程,以后一直是这三个进程在执行任务    res_l=[]    for i in range(10):        res=p.apply_async(work,args=(i,)) #同步运行,阻塞、直到本次任务执行完毕拿到res        res_l.append(res)    #异步apply_async用法：如果使用异步提交的任务，主进程需要使用jion，等待进程池内任务都处理完，然后可以用get收集结果，否则，主进程结束，进程池可能还没来得及执行，也就跟着一起结束了    p.close()    p.join()    # join之后get() 就能立即拿到值，如果注释掉上面两句代码，那么是每3个打印一次，没有结果的get会出现阻塞    for res in res_l:        print(res.get()) #使用get来获取apply_aync的结果,如果是apply,则没有get方法,因为apply是同步执行,立刻获取结果,也根本无需get

Callback

from multiprocessing import Poolimport os,timedef work(n):    time.sleep(1)    return n**2def get_data(data):    # 会把work return的结果传给get_data 做参数    print('得到的数据是：', data)if __name__ == '__main__':    p=Pool(os.cpu_count()) #进程池中从无到有创建三个进程,以后一直是这三个进程在执行任务    res_l=[]    # 回调函数是主进程执行的，如果主进程调用time.sleep(10000),当work执行完毕之后    # 操作系统就会跟主进程说，别睡了，快去处理任务，处理完再睡    for i in range(10):        p.apply_async(work,args=(i,), callback=get_data) #同步运行,阻塞、直到本次任务执行完毕拿到res    # 因为是异步的，如果不加下面两句，那么主进程退出，池子里的任务还没执行    p.close()    p.join()

The results are generally handled in two ways: one is to get the results immediately call, the other is to take the results to do a unified processing, the recommended use of the first way

Concurrent

import timefrom concurrent.futures import ProcessPoolExecutordef work(n):    time.sleep(1)    return n**2def get_data(res):    # 传的参数res是一个对象    print('得到的数据是：', res.result())if __name__ == '__main__':    executor = ProcessPoolExecutor(max_workers=5)    # futures = []    # for i in range(10):    #     future = executor.submit(work, i).add_done_callback(get_data)    #     futures.append(future)    # executor.shutdown()    # for future in futures:    #     print(future.result())    for i in range(10):        executor.submit(work, i).add_done_callback(get_data)    # 下面不加shutdown也能执行池里的任务，只不过为了代码可读性，一般还是建议加上

Map How to use

from concurrent.futures import ProcessPoolExecutorimport os,time,randomdef task(n):    print('%s is runing' %os.getpid())    time.sleep(random.randint(1,3))    return n**2if __name__ == '__main__':    executor = ProcessPoolExecutor(max_workers=3)    # for i in range(11):    #     # concurrent 模块不加shutdown 主进程执行完毕会等池里的任务执行完毕程序才会结束,不同于Pool    #     # 而且 concurrent 是用异步，没有Pool的同步方式    #     executor.submit(task,i)    # map 得到的是一个存储结果(不需要.result()) 的可迭代对象    data_obj = executor.map(task,range(1,12)) # map取代了for+submit    for data in data_obj:        print(data)

Greenlet and Gevent

The co-process in Python is Greenlet, which is the Toggle + save state. The thread is scheduled by the operating system, but the switch of the association is scheduled by the programmer, and the operating system is "invisible" to it. Since you want to save the state, then it will certainly involve the stack, the co-process also has its own stack, but this cost is smaller than the thread. The simple process can not help us improve efficiency, can only help us to save the last run state and to do the switch back and forth, so we are based on the switch + save state to do a further package, so that the program can encounter the IO blocking automatic switching, such as Gevent module. The Gevent module is implemented using the event-driven library Libev plus greenlet. We know that event loops are the underlying cornerstone of asynchronous programming. If the user pays attention to the level is very low, the direct operation Epoll constructs the maintenance event the cycle, the business logic from the bottom to the high level needs the layer callback, causes the callback hell, and the readability is poor. So, this cumbersome registration callback and callback process can be encapsulated and abstracted into eventloop. EventLoop masks the specific operation of making a epoll system call. For the user, considering the different I/O status as the trigger of the event, only focus on the callback behavior of the different events at a higher level. A high-performance asynchronous event library, such as Libev, Libevent, written in C, has replaced this trivial task. In summary, when the event occurs, notify the user program to switch the co-process. To be exact, the gevent is a third-party asynchronous module that allows me to use threads as a thread, and this module requires a socket that is non-blocking, so it is generally at the beginning of the program monkey.patch_all() .

import greenletdef task1():    print('task1 start')    g2.switch()    print('task1 end')    g2.switch()def task2():    print('task2 start')    g1.switch()    print('task2 end')g1 = greenlet.greenlet(task1)g2 = greenlet.greenlet(task2)g1.switch()

Basic use of the Gevent module

from gevent import spawn, joinall, monkeyimport timemonkey.patch_all()def task(i):    time.sleep(1)    print('----', i)    return i * 2if __name__ == '__main__':    # spawn 的时候会创建一个协程并立即执行    res = [spawn(task, i) for i in range(10)]    joinall(res)    for g in res:        print(g.value)

IO model user space and kernel space, user state and kernel state

Now that the operating system is using virtual memory, the 32-bit operating system, its addressing space (virtual storage space) is 4G (2 of 32). The core of the operating system is the kernel, which is independent of the normal application, has access to protected memory space, and has all the permissions to access the underlying hardware device. In order to ensure that the user process can not directly manipulate the kernel (kernel), to ensure the security of the kernel, worry about the system to divide the virtual space into two parts, part of the kernel space, part of the user space. For the Linux operating system, the highest 1G bytes (from the virtual address 0xc0000000 to 0xFFFFFFFF) for the kernel to use, called the kernel space, and the lower 3G bytes (from the virtual address 0x00000000 to 0xBFFFFFFF) for each process to use, Called User space.

The user state and the kernel state of the CPU, the user state is the CPU can execute less instruction, the kernel state is the CPU can handle more instructions, you can first so simple to understand.

synchronous, asynchronous, blocking, and non-blocking

The focus of both synchronous and asynchronous, blocking and non-blocking groups is different.
Synchronization is the programmer's program "personally" to take the initiative to wait for results, of course, in the process of such results can do other things (the process or the state of the thread is non-blocking), but the program will have a period of time to see if there is a result, the program needs to do two things, only one role
Async is the programmer's program to send a slogan to the results, and then wait for another "thing" to notify me that the result is good, this time the role of two. Of course, you can do other things in this process, or you can quit (unless you're stupid). The message notice here can also be seen as a callback,
Remember the main process above when the work task is completed and the operating system wakes up the main process to execute the callback function?
Blocking and non-blocking describe the state in which a process or thread is located.

Several different IO models

• Blocking IO
• Non-blocking IO
• Io Multiplexing (aka event-driven)
• Asynchronous IO

The four different IO models above are for the different states of the two phases: 1. Wait for data to kernel space 2. The process space of copying data from the kernel space to the user program

1、输入操作：read、readv、recv、recvfrom、recvmsg共5个函数，如果会阻塞状态，则会经理wait data和copy data两个阶段，如果设置为非阻塞则在wait 不到data时抛出异常2、输出操作：write、writev、send、sendto、sendmsg共5个函数，在发送缓冲区满了会阻塞在原地，如果设置为非阻塞，则会抛出异常3、接收外来链接：accept，与输入操作类似4、发起外出链接：connect，与输出操作类似

Complementary to IO multiplexing:

1. If the number of connections processed is not high, Web server using Select/epoll does not necessarily perform better than the Web server using multi-threading + blocking IO, and may be more delayed. The advantage of Select/epoll is not that a single connection can be processed faster, but that it can handle more connections.
2. In a multiplexed model, for each socket, it is generally set to non-blocking (our own program keeps going while looping to ask if the socket data is ready to be changed by the operating system to do this for us, So the general socket is set to non-blocking), however, as shown, the entire user's process is actually always block. Only the process is the block of the Select function, not the socket IO.

Select, poll, and Epool

These three technologies are developed by the ancients and are operating system-level program development.

1. Select: the least efficient, but with the maximum descriptor limit, in Linux 1024, a variety of platforms are supported. The operating system is a list of file objects that are monitored through a loop to see if there is a change in the state of the object, and if the first and last state of the list changes, then the front-to-back loop is needed to know that the two changes.
2. Poll: Same as SELECT, but no maximum descriptor limit.
3. Epoll: Highest efficiency, no maximum descriptor limit, support for horizontal triggering and edge triggering. Windows system is not supported. A change in the Epoll Monitor's socket notifies the operating system via a callback.

Two kinds of triggering modes in IO multiplexing:

1. Horizontal Trigger: If the file descriptor is ready to perform IO operations that are non-blocking, a notification is triggered. Allows the state of the IO to be repeated at any time, and it is not necessary to perform as many io.select,poll as possible after each descriptor is in the horizontal trigger.

2. Edge trigger: A notification is triggered if a file descriptor has a new IO activity since the last state change. To perform as many IO operations as possible after receiving an IO event notification, Because if you do not complete IO in one notification, you will need to wait until the next new IO activity arrives to get the ready descriptor. The signal-driven IO is an edge trigger.

   Selectors module

From socket import *import selectorssel=selectors. Defaultselector () def accept (Server_fileobj,mask): Conn,addr=server_fileobj.accept () Sel.register (conn,selectors. Event_read,read) def READ (conn,mask): Try:data=conn.recv (1024x768) if not data:print (' closing ', CO NN) sel.unregister (conn) conn.close () return Conn.send (Data.upper () +b ' _SB ') exc EPT exception:print (' closing ', conn) sel.unregister (conn) conn.close () Server_fileobj=socket (Af_inet, SOCK_STREAM) server_fileobj.setsockopt (sol_socket,so_reuseaddr,1) server_fileobj.bind ((' 127.0.0.1 ', 8088)) Server_ Fileobj.listen (5) server_fileobj.setblocking (False) #设置socket的接口为非阻塞sel. Register (server_fileobj,selectors. event_read,accept) #相当于网select的读列表里append了一个文件句柄server_fileobj, and binds a callback function Acceptwhile True:events=sel.select () # Detects all fileobj, if there is a for sel_obj,mask in Events:callback=sel_obj.data that completes wait data #callback =accpet callback ( Sel_obj.fileoBj,mask) #accpet (server_fileobj,1) #客户端from socket import *c=socket (af_inet,sock_stream) c.connect (' 127.0.0.1 ', 8088 ) while True:msg=input (' >>: ') If not msg:continue c.send (Msg.encode (' Utf-8 ')) Data=c.recv (1024x768) PRI NT (Data.decode (' Utf-8 '))

Problem solving for thread deadlock change the original multi-mutex to a repeatable lock

The mutex can only be acquire once, can be repeated acquire multiple times, here acquire multiple times for a single thread, if a thread acquire, within a thread can again acquire (internal maintenance of a counter), However, other threads cannot acquire

Multiple mutex locks in order to get

Each thread acquires a lock in a fixed order without problems, such as to acquire a lock, must follow the order of a lock after the B lock, then a thread acquires a lock, then another thread to acquire the lock must first take a lock, so as to solve the deadlock problem.

Import threadingimport timefrom contextlib Import contextmanager_local = Threading.local () # Unified interface, in the future to obtain a lock to use this function to obtain, This function makes a sort of lock @contextmanagerdef acquire (*locks): locks = sorted (locks, Key=lambda x:id (x)) # Here you save the thread with Threading.local () Local variables, mainly used for nesting to get the lock, # record the lock that the thread has acquired, and then compare it to the ID of the lock to get, the order does not conform to throw the exception acquired_locks = GetAttr (_local, ' acquired ', []) if Acq    Uired_locks and Max (ID (lock) for lock in acquired_locks) >= ID (locks[0]): Raise RuntimeError (' deadlock has occurred, program error exits ') Acquired_locks.extend (locks) _local.acquired = Acquired_locks try:for lock in Locks:lock.acquire () yield finally:for lock in reversed (locks): Lock.release () del Acquired_locks[-len (l ocks):]if __name__ = = ' __main__ ': X_lock = threading. Lock () Y_lock = Threading. Lock () # def thread_1 (): # with Acquire (X_lock, y_lock): # print (' Thread-1 ') # # def thread_2 () : # with Acquire (Y_lock, x_lock): # print (' Thread-2') # t1 = Threading. Thread (target=thread_1) # T1.start () # t2 = Threading.                Thread (target=thread_2) # T2.start () def thread_1 (): With Acquire (X_lock): With Acquire (Y_lock):                Print (' Thread-1 ') def thread_2 (): With Acquire (Y_lock): With Acquire (X_lock): Print (' Thread-2 ') T1 = threading. Thread (target=thread_1) t1.start () t2 = Threading. Thread (target=thread_2) T2.start ()

How async is implemented in Python

The starting point for mastering python async is: Epoll + callback + Event loop

Epoll

Determine if the non-blocking call is ready if the OS can do it, the application can use this idle to do other things to improve efficiency without having to wait and judge.
So the OS encapsulates the changes in I/O states into events, such as readable events and writable events. and a dedicated system module is provided to allow the application to receive event notifications. This module is select. Allows the application to register file descriptors and callback functions through select. When the state of the file descriptor changes, select invokes the pre-registered callback function.
Select because of its low efficiency algorithm, later improved into poll, and then further improved, the BSD kernel has been improved into the Kqueue module, and the Linux kernel has been improved into the Epoll module. The four modules have the same effect, and the APIs that are exposed to programmers are almost identical, except that Kqueue and epoll are more efficient at handling a large number of file descriptors.
The selectors module provided by the Python standard library is the encapsulation of the underlying select/poll/epoll/kqueue. The Defaultselector class automatically chooses the best modules based on the OS environment, which is epoll on Linux 2.5.44 and newer versions.

Callback (Callback)

The wait and listen tasks for I/O events are given to the OS, and the OS knows what to do next when it knows the I/O status has changed (for example, the socket connection has been established to send data). can only be recalled.
We need to encapsulate the sending data with the read data into a separate function, so that epoll instead of the application to listen to the socket state, you have to tell Epoll: "If the socket state becomes capable of writing data (connection established successfully), call the HTTP request send function. If the socket becomes readable data (the client has received a response), call the response handler function. The callback here is to notify the user of the process, not the operating system to execute the callback function.

Event Loops

This loop is the loop that our programmer writes in the program, not the Epoll loop of the operating system, and we go through this loop to access the Selector module and wait for it to tell us which event is currently occurring and which callback to correspond to. This loop, which waits for event notifications, is called an event loop.

Selector.select () is a blocking call, because if the event does not occur, the application will not be able to handle the event, so it simply blocks here to wait for the event to occur. It can be inferred that if you download only one page, be sure to connect () before send () and then recv (), it is the same efficiency and blocking way. Because it is not blocked on connect ()/recv (), it also has to be blocked on select (). Therefore, the selector mechanism is designed to solve a large number of concurrent connections. When there are a large number of non-blocking calls in the system, the selector mechanism can exert maximum power when the event can be generated at any time.

In some programming languages, support for asynchronous programming stops there (no extensions other than the official language). Requires the program ape to use Epoll to register events and callbacks, maintain an event loop, and spend most of the time on design callback functions.
No matter what programming language you want to do asynchronous programming, the "event loop + callback" mode above will not escape, although it may not use epoll, or it may not be a while loop. But the asynchronous approach used is basically the asynchronous way of "tell you later" models.

But why didn't you see CallBack mode in Asyncio async programming? Because Python asynchronous programming uses a process that helps us replace callbacks.

Python concurrent programming