Python multi-thread and queue management shell program, pythonshell

Python multi-thread and queue management shell program, pythonshell

First, describe the environment. There are many JAVA programs on the machine. We have configured a startup | stop | restart script in each JAVA program.

For example:

Now we need to run these scripts at the same time to quickly start all JAVA programs. If we only use multithreading, the thread will not return messages to the parent process, how can we know that these programs are successfully started?

So we use queues for management.

"I tried gevent, but it will cause blocking in command """

The gevent code is as follows. If you know how to optimize it, please let me know.

#! /Usr/bin/python2.7 #-*-coding: UTF-8-*-import OS, sysfrom datetime import datetimeimport commandsimport gevent. monkeygevent. monkey. patch_ OS () import gevent def Servers (): servers = commands. getoutput (''' ls/data/program/payment/| grep 'payment''') servers = servers. split ('\ n') return servers def handle (servername): if sys. argv [1] = 'start' or sys. argv [1] = 'stop' or sys. argv [1] = 'restart': print '\ 033 [1; 31; 40m 'print '======================================>> go to handle % s <= =====================================' % servername print '\ 033 [0m' r = commands. getoutput (''' su-tomcat-c "/data/program/payment/% s/bin/server. sh % s & "''' % (servername, sys. argv [1]) # It will be blocked here. We cannot find a suitable place to switch the gevent of the coroutine. sleep (0) # It will be blocked no matter where it is put. Print r else: print 'Please Use start | stop | restart To Handle The Command 'sys. exit (1) if _ name _ = '_ main _': s = Servers () threads = [] for I in s: threads. append (gevent. spawn (handle, I) # print threads gevent. joinall (threads)

The multi-threaded code is as follows:

#! /Usr/bin/python2.7 #-*-coding: UTF-8-*-from datetime import datetimeimport commandsfrom Queue import Queuefrom threading import Thread _ sentinel = object () Servers (): servers = commands. getoutput (''' ls/data/program/payment/| grep 'payment''') servers = servers. split ('\ n') return servers def producer (servername, out_q): if sys. argv [1] = 'start' or sys. argv [1] = 'stop' or sys. argv [1] = 'resta Rt ': print' \ 033 [1; 31; 40m 'print '====================================>> put % s in Queue <= ===================================' % servername print '\ 033 [0m' out_q.put_nowait (commands. getoutput (''' su-tomcat-c "/data/program/payment/% s/bin/server. sh % s & "''' % (servername, sys. argv [1]) # Put in The queue object else: print 'Please Use start | stop | restart To Handle The Command 'sys. exit (1) def consumer (servername, in_q): n = len (server Name) while n> 0: # cyclically obtain the result in the queue until the end of the Loop data = in_q.get () n-= 1 print '\ 033 [1; 31; 40m'print data print '\ 033 [0m'print' \ 033 [1; 31; 40m' print 'consumer was done !!!!!!! 'Print '\ 033 [0m' if _ name _ =' _ main _ ': s = Servers () q = Queue () t1 = Thread (target = consumer, args = (s, q,) # The consumer obtains the result in the queue. The previous function has cyclically obtained for I in s: t2 = Thread (target = producer, args = (I, q,) # manage threads and put them into the queue t2.start () # Start the producer Thread # t2.join () # after starting the producer, give up checking whether the thread is finished and run concurrently. Because we put the thread into the queue for management, we do not need to wait for the thread to end here, if join is used, our program will be blocked. After the thread ends, the consumer notifies the parent process that the thread has ended. T1.start () # Start consumer thread t1.join () # implement thread blocking before the collection is complete

To put it simply, join this method:

Calling Thread. join will block the main Thread until the call Thread ends or times out. The timeout parameter is a numeric value that indicates the timeout time. If this parameter is not provided, the main thread will be blocked until the end of the called thread.

The above is all the content of this article. I hope you will like it.