Python Concurrent Multi-threading

Source: Internet
Author: User

Concurrent series is a very large knowledge system, it is very difficult to fully understand, because the recent intention to read Tornado source code, the introduction of the internal use of asynchronous non-blocking call mode. I didn't know anything about it before, so I'll take a good look at it this time.

Thread Threading Module)

A thread is the smallest unit that the application runs, and in the same process, multiple threads can be opened concurrently, each with its own stack (which holds temporary variables), while sharing resources with each other.

Use the threading module in Python to turn on multithreading

Import threading, time def func (N):    time.sleep (2)
Print (Time.time (), N)
if __name__ ' __main__ '   for in range (Ten= threading. Thread (Target=func, args= (1print(' main thread end ')
Results
Main thread End
1532921321.058243 11532921321.058243 11532921321.058243 11532921321.058243 1 ...

or inherit the thread from the custom class, and override the Run method

ImportThreading,timeclassMythread (Threading. Thread):def __init__(Self,name): Super ().__init__() Self.name=namedefRun (self): Time.sleep (1)        Print('Hello%s'%self.name)if __name__=='__main__':     forIinchRange (5): M= Mythread ('py') M.start ()Print('---Main thread end---')

The execution sequence is as follows

The main thread and the child threads are independent of each other, but the main thread runs and waits for the child thread to run until it is complete before the resource is recycled.

methods that the thread object can invoke

methods  for Hread instance objects # isAlive (): Returns whether the thread is active.   #  getName (): Returns the thread name.   #  setName (): Sets the thread name.
#join (): Causes the main thread to block until the thread ends
Some of the methods provided by the threading module are: # threading.currentthread (): Returns the current thread variable. # threading.enumerate (): Returns a list that contains the running thread. Running refers to threads that do not include pre-and post-termination threads until after the thread has started and ends. # threading.activecount (): Returns the number of running threads with the same result as Len (Threading.enumerate ()).

daemon Thread (setdaemon)

If a thread is set to be a daemon thread, it will end with the main thread, and the main thread will wait for all the non-daemon threads to end and exit. Therefore, it can be considered that the daemon thread is "unimportant thread" and the main thread is not equal to it.

Importthreading, time"""set two threads"""deffunc1 ():Print('-- non-daemon thread start- -') Time.sleep (2)    Print('-- end of non-daemon thread--')defFunc2 ():Print('--Daemon thread start- -') Time.sleep (4)    Print('-- Daemon thread end--')if __name__=='__main__': T1= Threading. Thread (target=func1,args=()) T2= Threading. Thread (target=func2,args=()) T2.setdaemon (True) T1.start () T2.start ()'''--Non-daemon thread starts----daemon thread starts----non-daemon thread ends-the main thread ends when the daemon is not running yet'''

The sharing of data between threads inevitably leads to confusion at the same time when manipulating data, affecting data security

ImportThreading,timedeffunc ():#Start processing Data    GlobalN a=n+1Time.sleep (0.0001) n=a#End Processingif __name__=='__main__': N=0 Li=[]     forIinchRange (1000): T=threading. Thread (target=func,args=()) li.append (t) t.start () forIinchLi:i.join ()#wait for all child threads to finish executing    Print(n)#253    '''we want to be able to add from 0 to 1000, but because there are multiple threads that will get the data, if the processing speed is slow, it will confuse the data'''

Therefore, it is necessary to lock the data.

Mutex (lock)

By acquiring the lock object, accessing the common data, and finally releasing the lock to complete one operation, once a thread acquires the lock, when the thread is switched, the next process cannot get the public data

ImportThreading,timedeffunc ():#Start processing Data    Globaln Lock.acquire () #获取 a=n+1Time.sleep (0.00001) n=a lock.release () #释放#End Processingif __name__=='__main__': N=0 Lock=Threading. Lock () Li=[]     forIinchRange (1000): T=threading. Thread (target=func,args=()) li.append (t) t.start () forIinchLi:i.join ()#wait for all child threads to finish executing    Print(n)# +

By synchronizing the mutex to ensure data security compared to thread serial operation, such as each thread start before the use of the. Join () method, is undoubtedly faster, because it is only when the data is accessed at the time of the Cascade, the other case is concurrent (although it can not be run in parallel, because Gil, will be discussed later).

Let's look at the following situation

dead Lock

Import Threading if __name__ ' __main__ ' :    Lock=Threading. Lock ()    lock.acquire (    )# program stuck, can only get once     lock.release ()    Lock.release ()

Recursive lock (Rlock)

Rlock internally maintains a lock and a counter variable, counter records the number of acquire, so that resources can be require multiple times. Until all the acquire of a thread are release, the other threads can get the resources. The above example if the

With Rlock instead of lock, no deadlock occurs:

Import Threading if __name__ ' __main__ ' :    Lock=Threading. Rlock ()    lock.acquire (    )# can be obtained multiple times, the program executes smoothly     lock.release ()    Lock.release ()
   

Signal Volume (Semaphore)

The number of threads that can execute concurrently, and the thread that is out of the way is blocked until the wired thread runs.

Semaphore manages a built-in counter,
Built-in counter whenever acquire () is called-1;
Built-in counter +1 when call Release ();
The counter cannot be less than 0, and when the counter is 0 o'clock, acquire () blocks the thread until another thread calls release ().

ImportThreading,timedefsever_help (N): S.acquire ()Print('%s Welcome User%d'%(Threading.current_thread (). GetName (), N)) Time.sleep (2) s.release ()if __name__=='__main__': S= Threading. Semaphore (5) Li= []     forIinchRange (32): T= Threading. Thread (target=sever_help,args=(i,)) Li.append (t) t.start () forIinchLi:i.join ()Print("= = = End = =")

By comparing the mutex, it can be seen that the mutex is the case of Semaphore (1) and the latter can be used completely, but the mutex is more efficient if the data must be used alone.

Events (Event)

If a thread executes and needs to determine the state of another thread, it can use the event, such as initializing an event object with the event class, thread a executing to a step, setting event.wait (), which is thread a blocking until another thread sets Event.set (), The event

The status is set to True (the default is False).

ImportThreadingImportTime , Randomdefeating (): event.wait ()Print('on the way to dinner ...')defmakeing ():Print('in the cooking') Time.sleep (Random.randint (The))    Print('do it, come on ...') Event.set ()if __name__=='__main__': Event=Threading. Event () T1= Threading. Thread (target=eating) T2= Threading. Thread (target=makeing) T1.start () T2.start ()#in the cooking    #do it, come on ...    #on the way to dinner ...
I'm not going to eat until dinner is ready.

Basic methods:

event.isset (): Returns the status value of the event, event.wait (): If Event.isset ()= =false will block the thread; Event.set (): Sets the status value of event to True, All blocking pool threads are activated into a ready state, waiting for the operating system to dispatch; Event.clear (): The status value of recovery event is false. 

Thread queues (queue)

Features: Advanced First Out,

Role: Communication between multiple threads (small, multi-process queues are useful)

Common methods:

    • . Get () blocks when getting no data
    • . Set (' Item ') settings, first set the data, first take out
    • The. Empty () is empty

Basic use: Creator consumer model

Importthreading, Queuedefeating (n):#Consumeri =Q.get ()Print('consumer%d ate the first%d servings of food'%(n, i))defmaking ():#Production     forIinchRange (1, 11):        Print('working on the production of part%d food'%i) Time.sleep (1) Q.put (i)if __name__=='__main__': Q=queue. Queue () T2= Threading. Thread (target=makeing) T2.start () forIinchRange (1, 11): T= Threading. Thread (target=eating, args=(i,)) T.start ()
producers, consumers

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: info-contact@alibabacloud.com and provide relevant evidence. A staff member will contact you within 5 working days.

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.