Example of deadlock formation and prevention of deadlock in Python, python example

Example of deadlock formation and prevention of deadlock in Python, python example

Deadlock example
We often encounter deadlocks when dealing with multiple threads. When learning the operating system, we will talk about deadlocks. We will use Python to demonstrate them intuitively.
One reason for the deadlock is the mutex lock. Assume that user a tries to transfer 100 yuan to user B in the banking system, and user B tries to transfer 200 yuan to user a, a deadlock may occur.
The two threads wait for each other's lock and occupy each other's resources.

# Coding = UTF-8 import time import threading class Account: def _ init _ (self, _ id, balance, lock): self. id = _ id self. balance = balance self. lock = lock def withdraw (self, amount): self. balance-= amount def deposit (self, amount): self. balance + = amount def transfer (_ from, to, amount): if _ from. lock. acquire (): # Lock your account _ from. withdraw (amount) time. sleep (1) # Extend the transaction time, overlap the time of two transaction threads, and have enough time to generate the deadlock print 'Wait for lock... 'If. lock. acquire (): # Lock the account. deposit (amount). lock. release () _ from. lock. release () print 'finish... 'A = Account ('A', 1000, threading. lock () B = Account ('B', 1000, threading. lock () threading. thread (target = transfer, args = (a, B, 100 )). start () threading. thread (target = transfer, args = (B, a, 200 )). start ()

Lock mechanism to prevent deadlocks
Problem:
You are writing a multi-threaded program, where the thread needs to obtain multiple locks at a time, how to avoid the deadlock problem at this time.
Solution:
In multi-threaded programs, most of the deadlock problems are caused by the simultaneous acquisition of multiple locks by threads. For example, if a thread acquires the first lock and blocks the second lock, the thread may block the execution of other threads, causing the entire program to be suspended. One solution to the deadlock problem is to assign a unique id for each lock in the program, and then only allow multiple locks to be used in ascending order, this rule can be easily implemented using the context manager, for example:

import threadingfrom contextlib import contextmanager# Thread-local state to stored information on locks already acquired_local = threading.local()@contextmanagerdef acquire(*locks):  # Sort locks by object identifier  locks = sorted(locks, key=lambda x: id(x))  # Make sure lock order of previously acquired locks is not violated  acquired = getattr(_local,'acquired',[])  if acquired and max(id(lock) for lock in acquired) >= id(locks[0]):    raise RuntimeError('Lock Order Violation')  # Acquire all of the locks  acquired.extend(locks)  _local.acquired = acquired  try:    for lock in locks:      lock.acquire()    yield  finally:    # Release locks in reverse order of acquisition    for lock in reversed(locks):      lock.release()    del acquired[-len(locks):]

How to Use the context manager? You can create a lock object in the normal way, but use the acquire () function to apply for a lock in either a single lock or multiple locks. The example is as follows:

import threadingx_lock = threading.Lock()y_lock = threading.Lock()def thread_1():  while True:    with acquire(x_lock, y_lock):      print('Thread-1')def thread_2():  while True:    with acquire(y_lock, x_lock):      print('Thread-2')t1 = threading.Thread(target=thread_1)t1.daemon = Truet1.start()t2 = threading.Thread(target=thread_2)t2.daemon = Truet2.start()

If you execute this code, you will find that it does not get the lock in different orders in different functions. The key is that in the first code, we sorted these locks. By sorting, the locks are obtained in a fixed order regardless of the order in which the user requests the locks. If multiple acquire () operations are nested and called, you can detect potential deadlock problems through local thread storage (TLS. Assume that your code is written as follows:

import threadingx_lock = threading.Lock()y_lock = threading.Lock()def thread_1():  while True:    with acquire(x_lock):      with acquire(y_lock):        print('Thread-1')def thread_2():  while True:    with acquire(y_lock):      with acquire(x_lock):        print('Thread-2')t1 = threading.Thread(target=thread_1)t1.daemon = Truet1.start()t2 = threading.Thread(target=thread_2)t2.daemon = Truet2.start()

If you run the code of this version, a thread will crash and the exception information may look like this:

Exception in thread Thread-1:Traceback (most recent call last): File "/usr/local/lib/python3.3/threading.py", line 639, in _bootstrap_inner  self.run() File "/usr/local/lib/python3.3/threading.py", line 596, in run  self._target(*self._args, **self._kwargs) File "deadlock.py", line 49, in thread_1  with acquire(y_lock): File "/usr/local/lib/python3.3/contextlib.py", line 48, in __enter__  return next(self.gen) File "deadlock.py", line 15, in acquire  raise RuntimeError("Lock Order Violation")RuntimeError: Lock Order Violation>>>

The cause of the crash is that every thread records the locks it has obtained. The acquire () function checks the list of previously obtained locks because they are obtained in ascending order, therefore, the function will think that the id of the previously obtained lock must be smaller than the newly applied lock, and an exception will be triggered.

Discussion
Deadlock is a problem that every multi-threaded program will face (just as it is a common topic of every operating system textbook ). Based on experience, try to ensure that each thread can only maintain one lock at a time, so that the program will not be troubled by the deadlock problem. Once a thread applies for multiple locks at the same time, everything is unpredictable.

Deadlock Detection and recovery is an extended topic with almost no elegant solutions. A common Deadlock Detection and recovery solution is to introduce the watchdog counter. When the thread runs normally, the counter is reset at intervals. In the absence of a deadlock, everything works normally. In the event of a deadlock, the timer times out because the counter cannot be reset. In this case, the program restarts itself and returns to normal.

Preventing deadlocks is another way to solve the deadlock problem. When the process acquires the lock, it will obtain the lock in ascending order according to the Object id. After mathematical proof, the program will not enter the Deadlock State. The proof is left to the reader as an exercise. The main idea of avoiding deadlocks is that simply locking in the order of increasing object IDs will not generate circular dependencies, and circular dependencies are a necessary condition for deadlocks, so as to prevent the program from entering the deadlocked state.

The following describes a typical thread deadlock problem: "Dining Philosophers", which is the last example in this section. The question is as follows: five philosophers sat in front of a table, with a bowl of rice and chopsticks in front of each person. Here, each philosopher can be regarded as an independent thread, and each chopsticks can be regarded as a lock. Every philosopher can be in one of three states: sit-in, think, and eat. It should be noted that every philosopher needs two chopsticks to eat, so the problem arises: if every philosopher picks up the chopsticks on his left, then the five of them can only sit there with one chopsticks until they starve to death. Then they enter the Deadlock State. The following is a simple implementation of using the deadlock avoidance mechanism to solve the "Dining Philosophers problem:

import threading# The philosopher threaddef philosopher(left, right):  while True:    with acquire(left,right):       print(threading.currentThread(), 'eating')# The chopsticks (represented by locks)NSTICKS = 5chopsticks = [threading.Lock() for n in range(NSTICKS)]# Create all of the philosophersfor n in range(NSTICKS):  t = threading.Thread(target=philosopher,             args=(chopsticks[n],chopsticks[(n+1) % NSTICKS]))  t.start()

Finally, it should be noted that in order to avoid deadlocks, all locking operations must use the acquire () function. If a part of the Code directly applies for a lock by bypassing the acquire function, the entire Deadlock Avoidance Mechanism does not work.