Multiple python threads and concurrent execution of multiple commands

Multiple python threads and concurrent execution of multiple commands

I. Introduction to concepts

Thread is one of the most important classes in the threading module and can be used to create threads. There are two ways to create a Thread: one is to inherit the Thread class and override its run method; the other is to create a threading. thread object. In its initialization function (_ init _), the object can be called as a parameter.
The Thread module is a relatively low-level module, and the Threading module packages the Thread and can be used more conveniently.
In addition, at work, it is sometimes necessary to allow concurrent execution of multiple commands, rather than sequential execution.

Ii. Sample Code

#! /Usr/bin/python # encoding = UTF-8 # Filename: thread-extends-class.py # inherit from Thread directly, create a new class, put the code executed by the thread into this new class. import threadingimport time class ThreadImpl (threading. thread): def _ init _ (self, num): threading. thread. _ init _ (self) self. _ num = num def run (self): global total, mutex # print the thread name print threading. currentThread (). getName () for x in xrange (0, int (self. _ num): # obtain the lock mutex. acquire () total = total + 1 # Release the lock mutex. release () if _ name _ = '_ main _': # defines the global variable global total, mutex total = 0 # creates the lock mutex = threading. lock () # define thread pool threads = [] # create thread object for x in xrange (0, 40): threads. append (ThreadImpl (100) # Start the thread for t in threads: t. start () # Wait until the sub-thread ends for t in threads: t. join () # print the execution result print total

#! /Usr/bin/python # encoding = UTF-8 # Filename: thread-function.py # create the function to be executed by the Thread, pass this function into the Thread object, run import threadingimport time def threadFunc (num): global total, mutex # print the thread name print threading. currentThread (). getName () for x in xrange (0, int (num): # obtain the lock mutex. acquire () total = total + 1 # Release the lock mutex. release () def main (num): # define the global variable global total, mutex total = 0 # create the lock mutex = threading. lock () # define thread pool threads = [] # first create a thread object for x in xrange (0, num): threads. append (threading. thread (target = threadFunc, args = (100,) # Start all threads for t in threads: t. start () # Wait for all sub-threads to exit in the main thread for t in threads: t. join () # print the execution result print total if _ name _ = '_ main _': # create 40 threads main (40)

#! /Usr/bin/python # encoding = UTF-8 # Filename: put_files_hdfs.py # concurrent execution of multiple commands, such as concurrent execution of multiple scp, ftp, and hdfs upload commands, improve program running efficiency import datetimeimport osimport threadingdef execCmd (cmd): try: print "command % s start running % s" % (cmd, datetime. datetime. now () OS. system (cmd) print "command % s end run % s" % (cmd, datetime. datetime. now () failed t Exception, e: print '% s \ t failed to run, cause of failure \ r \ n % s' % (cmd, e) if _ name _ = '_ main _': # list of commands to be executed cmds = ['ls/root', 'pwd',] # threads = [] print "program starts running % s" % datetime. datetime. now () for cmd in cmds: th = threading. thread (target = execCmd, args = (cmd,) th. start () threads. append (th) # Wait for the thread to finish running for th in threads: th. join () print "program stops running % s" % datetime. datetime. now ()

The above is all the content of this article. I hope it will help you learn python programming.