Introduction to methods of non-forced termination of Python threads

This article is to share with you some of the lessons that are going to be forced to kill Python threads, and if you use coercion to get rid of threads, there is a big chance of unexpected bugs. Keep in mind that lock resources do not release lock resources because of thread exits!

Objective:

Do not attempt to kill a python thread with a forced method, which is unreasonable from the service design. Multithreading is used to collaborate on tasks, and if you use coercion to kill threads, there is a big chance of unexpected bugs. Keep in mind that lock resources do not release lock resources because of thread exits!

We can cite two common examples:

1. A thread got a lock, because he was forced to kill, failed to release the lock resources in time, so that all the threads to get the resources are blocked, this is the typical deadlock scenario.

2. In the scenario of a common production consumer, the consumer gets the task from the task queue, but does not throw the task back into the queue after being killed, which results in data loss.

Here's how Java and python Terminate threads:

There are three ways Java can make a terminating thread:

1. Use the exit flag to cause the thread to exit normally, that is, the thread terminates when the Run method completes.
2. Use the Stop method to forcibly terminate a thread (deprecated, because stop and suspend, resume, can also cause unpredictable results).
3. Use the interrupt method to break the thread.

Python can have two methods:

1. Exit marker
2. Forcibly kill a thread using cTYPES

In both Python and Java environments, the ideal way to stop exiting a thread is to let the thread self-commit suicide, so-called thread suicide is you give him a flag bit and he quits the thread.

Here are a few ways to test the exception that stops the Python thread. We look at all the execution threads of a process, the process is used to control the resources, the thread is used as a dispatch unit, the process to be scheduled to execute must have a thread, the default thread and process PID.

PS-MP 31449-o thread,tid user   %cpu PRI SCNT wchan user SYSTEM  tidroot   0.0  -  -   --   -ro OT   0.0-  poll_s  -   -31449root   0.0-  poll_s  -   -31450

When we get to all the threads in the process, we know by Strace that 31450 is the thread ID that needs our kill, and when we kill, the whole process crashes. In a multithreaded environment, the resulting signal is passed to the entire process, in general, all threads have the opportunity to receive this signal, the process in the receiving signal of the thread context to execute the signal processing function, specifically which thread execution is difficult to learn. That is, the signal is randomly sent to a thread of the process.

Strace-p <span style= "font-size:14px;line-height:21px;" >31450</span> Process <span style= "font-size:14px;line-height:21px;" >31450</span> Attached-interrupt to Quitselect (0, NULL, NULL, NULL, {0, 320326}) = 0 (Timeout) Select (0, NULL, N ULL, NULL, {1, 0})   = 0 (timeout) select (0, NULL, NULL, NULL, {1, 0})   = 0 (Timeout) Select (0, NULL, NULL, NULL, {1, 0 })   = ? Erestartnohand (to restarted)---SIGTERM (Terminated) @ 0 (0)---Process <span style= "font-size:14px;line-height:21 px; " >31450</span> detached

The above problems are in fact consistent with Pthread's instructions. When we add the signal signal processing function to the Python code, the callback function prevents the entire process from exiting, so the problem is that the signal function does not recognize which thread you want to kill, that is, you cannot kill a thread precisely. Although you send the signal to the 31450 thread ID, but the signal handler is any one of the process, and the signal processing function is transmitted to the parameter only signal number and signal stack, optional.

Process is not exited after signal processing has been added

Select (0, NULL, NULL, NULL, {1, 0})   = 0 (Timeout) Select (0, NULL, NULL, NULL, {1, 0})   =? Erestartnohand (to restarted)---SIGTERM (Terminated) @ 0 (0)---rt_sigreturn (0xffffffff)        = 1 Eintr (interrupted s Ystem Call) Select (0, NULL, NULL, NULL, {1, 0})   = 0 (timeout) select (0, NULL, NULL, NULL, {1, 0})   = 0 (Timeout)

If you want to kill a thread from an external notification, you can build using the RPC service, or otherwise communicate, and the signal signal is not possible because more information cannot be delivered.

The python thread is not analog, it is the real kernel thread, the kernel calls the Pthread method, but the Python upper layer does not provide a way to close the thread, which we need to grasp. It is strongly recommended that you use the event or custom flag bit method, and if you want to force the thread to be killed, you can force exit with the Python ctypes pythreadstate Setasyncexc method, which has no effect on the running Python service.

The implementation of this function is relatively simple, but also in the Python virtual machine to make a mark, and then by the virtual machine run an exception to cancel the thread, virtual opportunity to help you do the try cache. Remember not to kill one of the python threads outside, although you can find the thread ID through ctypes, but you kill the whole process directly.

The following code is an example of using cTYPES to kill a thread, which is not recommended because it is too rough.

Import ctypes def terminate_thread (thread):  if not thread.isalive ():    return   exc = Ctypes.py_object ( Systemexit)  res = CTYPES.PYTHONAPI.PYTHREADSTATE_SETASYNCEXC (    ctypes.c_long (thread.ident), exc)  if res = = 0:    raise ValueError ("Nonexistent thread id")  elif res > 1:    ctypes.pythonapi.PyThreadState_ Setasyncexc (Thread.ident, None)    raise Systemerror ("Pythreadstate_setasyncexc failed")

Let's simply look at the Pythreadstate source code, which in summary triggers the thread's exception pattern. Interested people can read the Python pystate.c design, with some YouTube video sharing.

INTPYTHREADSTATE_SETASYNCEXC (Long ID, pyobject *exc) {  pyinterpreterstate *interp = Get_interp_state ();  ...  Head_lock ();  for (p = interp->tstate_head; p! = NULL; p = p->next) {    if (p->thread_id = = ID) {The id of the      thread is found from the list, avoiding deadlocks, we need to release Head_mutex.      pyobject *old_exc = p->async_exc;      Py_xincref (EXC); #增加该对象的引用数      p->async_exc = exc; # more exc mode      head_unlock ();      Py_xdecref (OLD_EXC); # because to cancel, of course also decrements the reference      ...      return 1; #销毁线程成功    }  }  head_unlock ();  return 0;}

The native POSIX pthread can use Ptread_cancel (TID) to end a child thread in the main thread. But Python's line libraries does not support this because we should not force the end of a thread, which poses many pitfalls and should let the thread end itself. So in Python, the recommended method is to iterate over a flag bit in a child thread, change the flag bit in the main thread, and the sub-thread reads the flag bit to change and ends itself.

Similar to this logic:

Def consumer_threading (): t1_stop= Threading. Event () T1 = Threading. Thread (Target=thread1, args= (1, t1_stop))  T2_stop = Threading. Event () t2 = Threading. Thread (Target=thread2, args= (2, t2_stop))  time.sleep (duration) #stop the Thread2 t2_stop.set () def thread1 (Arg1, Stop_event): while (not Stop_event.is_set ()):   #similar to Time.sleep ()   stop_event.wait (time)   pass  def thread2 (Arg1, stop_event): while (not Stop_event.is_set ()):   stop_event.wait (time)   Pass

Simple summary, although we can use the ctypes in the Pystats to control the thread, but this rough in the method of disconnection process is unreasonable. Please choose Suicide mode! What happens if your thread is blocking IO and can't tell what happened? Your program needs to be optimized, at least in the network IO layer need to have an active timeout, to avoid blocking all the time.

"Recommended"

1. Python Free video tutorial

2. Python Basics Getting Started tutorial

3. Python Object-oriented video tutorial