Python tips: time-out decorator, @ timeout decorator

I have been working a little more recently. I will continue to share some tips on how to learn and use python over the weekend.

Have you ever encountered such a problem: performing the insert or update operation on the database, because of the large amount of data or other reasons, this operation is very time-consuming, sometimes even waiting for several hours, cannot be completed. Very depressing. How can I avoid timeout? Because the database configuration time-out period is long and some operations take a long time, the default time-out period cannot be modified.

Because objective conditions are not allowed, we cannot terminate this operation because of database timeout. Therefore, we must implement the timeout detection function in our method logic module.

Can I implement timeout detection in python without modifying the internal logic of the original method? You must have it, that is, using the decorator. What is the decorator? In the blog, I found an article about Function decorator ).

After all this talk, I will introduce the leading role: time-out decorator and timeout decorator.

Timeout detection logic: Start a new sub-thread to execute the specified method. The main thread waits for the running result of the sub-thread. If the sub-thread has not completed execution within the specified time, it is deemed as timeout, throw a timeout exception and kill the sub-thread. Otherwise, the return value of the method executed by the sub-thread is returned.

In the implementation process, it is found that there is no way to kill the thread in the python default module. What should I do? Of course, first ask google or Baidu. Sure enough, the keyword keill thread is very popular and soon I found what I want: "Kill a thread in Python ", is the following KThread class, which inherits threading. thread, and added the kill method so that we can kill it:

 

Import sys

Class KThread (threading. Thread ):
"A subclass of threading. Thread, with a kill ()
Method.

Come from:
Kill a thread in Python:
Http://mail.python.org/pipermail/python-list/2004-May/260937.html
"""
Def _ init _ (self, * args, ** kwargs ):
Threading. Thread. _ init _ (self, * args, ** kwargs)
Self. killed = False

Def start (self ):
"Start the thread ."""
Self. _ run_backup = self. run
Self. run = self. _ run # Force the Thread to install our trace.
Threading. Thread. start (self)

Def _ run (self ):
"Hacked run function, which INSTALLThe
Trace ."""
Sys. settrace (self. globaltrace)
Self. _ run_backup ()
Self. run = self. _ run_backup

Def globaltrace (self, frame, why, arg ):
If why = 'call ':
Return self. localtrace
Else:
Return None

Def localtrace (self, frame, why, arg ):
If self. killed:
If why = 'line ':
Raise SystemExit ()
Return self. localtrace

Def kill (self ):
Self. killed = True

 

Well, everything is on guard. let's complete the remaining code, that is, timeout decorator:

Class Timeout (Exception ):
"Function run timeout """

Def timeout (seconds ):
"Time-out decorator, specifying the time-out period
If the decorated method is not returned within the specified time, a Timeout exception "" is thrown """
Def timeout_decorator (func ):
"Real decorators """

Def _ new_func (oldfunc, result, oldfunc_args, oldfunc_kwargs ):
Result. append (oldfunc (* oldfunc_args, ** oldfunc_kwargs ))

Def _ (* args, ** kwargs ):
Result = []
New_kwargs = {# create new args for _ new_func, because we want to get the func return val to result list
'Old': func,
'Result': result,
'Oldfunc _ args ': args,
'Oldfunc _ kwargs ': kwargs
}
Thd = KThread (target = _ new_func, args = (), kwargs = new_kwargs)
Thd. start ()
Thd. join (seconds)
Alive = thd. isAlive ()
Thd. kill () # kill the child thread
If alive:
Raise Timeout (u'function run too long, timeout % d seconds. '% seconds)
Else:
Return result [0]
_. _ Name _ = func. _ name __
_. _ Doc _ = func. _ doc __
Return _
Return timeout_decorator

 

Can it really run? Write a test program to run:

@ Timeout (5)
Def method_timeout (seconds, text ):
Print 'start', seconds, text
Time. sleep (seconds)
Print 'finish ', seconds, text
Return seconds

If _ name _ = '_ main __':
For sec in range (1, 10 ):
Try:
Print '* 20
Print method_timeout (sec, 'test waiting % d seconds '% sec)
Timeout t Timeout, e:
Print e

 

Look, the real line:

 

 

Original code: threadutil.py.zip

 

This tips may be a bit complicated. It applies to multithreading, decorator, etc. I hope it will help you.

PS: The scenery on the top of the hill is really top-notch, And the beauty is lacking !!!