Getting started with Python learn-day36-gil global interpreter locks, deadlock phenomena and recursive locks, semaphores, event events, thread queue

One, Gil Global interpreter lock 1. What is the Gil Global interpreter lock

The Gil is essentially a mutex, equivalent to execute permissions

Under the CPython interpreter, if you want to implement parallelism, you can open multiple processes

2. Why should there be Gil

The first thing we need to know is how a multithreaded execution, assuming that there are three threads in a process, is the code to run in the thread.

① If you want to run the code, you must first obtain the permissions of the CPython interpreter to translate the code into a language that the CPU can understand.

② the translated code to the operating system, the operating system to the CPU to perform the operation.

Since there will be a Gil within each process, multiple threads within the same process must be robbed of the Gil before using the CPython interpreter to execute their own code

That is, multiple threads under the same process cannot implement parallelism, but can implement concurrency

Then we think about the other hand, if there is no Gil, then multiple threads will become parallel, to know that the interpreter has a garbage collection mechanism, is actually a thread, also become parallel, will cause a situation, for the same data 100, may thread 1 execution x=100, Garbage collection performs a 100 recovery operation, resulting in loss of data.

To CPython the garbage collection mechanism of the interpreter, you must use the Gil

3. How to use Gil

With Gil, what should be done with concurrency

We have four tasks to deal with, and the way to do this is to play the concurrency effect, and the solution can be:

Scenario One: Open four processes

Scenario Two: Open four threads under one process

Single-core case, the results of the analysis:

If four tasks are computationally intensive and there is no multicore for parallel computing, the scenario increases the cost of creating the process, and the scheme wins

If the four tasks are I/O intensive, the cost of the scenario one creation process is large, and the process is much slower than the thread, the scheme wins

Multi-core scenario, the results of the analysis:

If four tasks are computationally intensive, multicore means parallel computing, where only one thread executes at the same time in Python in a process with no more cores, the scheme wins

If the four tasks are I/O intensive, and no more cores can solve the I/O problem, the scheme wins

Conclusion: The current computer is mostly multicore, Python for the computation-intensive task of multi-threading efficiency does not bring much performance improvement, or even less than serial (no large switching), but for the IO-intensive task efficiency is significantly improved.

COMPUTE-intensive

 fromMultiprocessingImportProcess fromThreadingImportThreadImportOs,timedefTask (): Res=0 forIinchRange (100000000): Res*=Iif __name__=='__main__': L=[]    Print(Os.cpu_count ())#This machine is 4 coresstart=time.time () forIinchRange (4):        #p=process (Target=task) #耗时16.226743459701538sP=thread (Target=task)#Time consuming 26.44382882118225sl.append (P) p.start () forPinchl:p.join () Stop=time.time ()Print('run time is%s'% (Stop-start))

View CodeI/O intensive

 fromMultiprocessingImportProcess fromThreadingImportThreadImportOs,timedeftask (): Time.sleep (2)if __name__=='__main__': L=[]    Print(Os.cpu_count ())#This machine is 4 coresstart=time.time () forIinchRange (400):        #p=process (Target=task) #耗时29.650749683380127sP=thread (Target=task)#Time consuming 2.0773582458496094sl.append (P) p.start () forPinchl:p.join () Stop=time.time ()Print('run time is%s'% (Stop-start))

View Code

Two, deadlock phenomenon and recursive lock deadlock phenomenon

That is, thread 1 gets the lock that thread 2 needs, thread 2 takes the lock that thread 1 needs, both sides hold the resource that each other needs, but neither side can release

Like me I was locked in this room, my hand was holding the key to the next room, and the other man was locked in the room next door, and he was holding the key to my room, and we all needed the key, but it was locked.

This is the deadlock, refers to two or more than two processes or threads in the execution process, because of the contention for resources caused by a mutual waiting phenomenon, if there is no external force, they will not be able to proceed.

 fromThreadingImportThread,lockImportTimemutexa=Lock () Mutexb=Lock ()classMythead (Thread):defRun (self): Self.f1 () self.f2 ( )defF1 (self): Mutexa.acquire ()Print('%s Grab the A lock'%self.name) Mutexb.acquire ()Print('%s grab the B lock'%self.name) mutexb.release () mutexa.release ()defF2 (self): Mutexb.acquire ()Print('%s grabbed the B lock.'%self.name) Time.sleep (2) Mutexa.acquire ()Print('%s grabbed a lock.'%self.name) mutexa.release () mutexb.release ()if __name__=='__main__':     forIinchRange (100): T=Mythead () T.start ( )

View CodeRecursive lock

In Python, in order to support multiple requests for the same resource in the same thread, Python provides a reentrant lock rlock.

The Rlock internally maintains a lock and a counter variable, counter records the number of acquire, so that resources can be require multiple times. Until all the acquire of a thread are release, the other threads can get the resources. In the example above, if you use Rlock instead of lock, a deadlock will not occur

 fromThreadingImportThread,lock,rlockImportTimemutexb=mutexa=Rlock ()classMythead (Thread):defRun (self): Self.f1 () self.f2 ( )defF1 (self): Mutexa.acquire ()Print('%s Grab the A lock'%self.name) Mutexb.acquire ()Print('%s grab the B lock'%self.name) mutexb.release () mutexa.release ()defF2 (self): Mutexb.acquire ()Print('%s grabbed the B lock.'%self.name) Time.sleep (2) Mutexa.acquire ()Print('%s grabbed a lock.'%self.name) mutexa.release () mutexb.release ()if __name__=='__main__':     forIinchRange (100): T=Mythead () T.start ( )

View Code

Three, the signal volume

Semaphore semaphore is also allowed a certain number of threads to change data, such as the toilet has 5 pits, the maximum allowed only 5 people to the toilet, the back of the people can only wait inside someone out to go in, if the specified signal volume is 5, then a person to obtain a lock, Count plus 1, when the Count equals 5 o'clock, Everyone in the back needs to wait. Once released, someone can get a lock

 fromThreadingImportThread,semaphoreImporttime,randomsm=semaphore (5)defTask (name): Sm.acquire ()Print('%s is on the toilet'%name) time.sleep (Random.randint (1,3) ) sm.release ()if __name__=='__main__':     forIinchRange (20): T=thread (target=task,args= ('passer -by%s'%I,)) T.start ()

View Code

Iv. Event Events

When a thread needs to be judged by another thread, such as a traffic light junction, if the vehicle can travel, it relies on the information given by the traffic lights to travel. Event events are required to solve this problem

Event.isset (): Returns the status value of the event;

Event.wait (): If Event.isset () ==false will block the thread;

Event.set (): Sets the status value of event to true, all the threads of the blocking pool are activated into a ready state, waiting for the operating system to dispatch;

Event.clear (): The status value of recovery event is false.

 fromThreadingImportthread,eventImporttimeevent=Event ()defLight ():Print('The red light is on.') Time.sleep (3) Event.set ()#green lightdefcar (name):Print('car%s is waiting for green light'%name) event.wait ()#wait for the light green    Print('Car%s Pass'%name)if __name__=='__main__':    #Traffic LightsT1=thread (target=Light ) T1.start ()#Car     forIinchRange (10): T=thread (target=car,args=(i,)) T.start ()

View Code

Five, Thread queue

FIFO queue. Queue ()

Import Queueq=queue. Queue (3) q.put (1) q.put (2) q.put (3)print(Q.get ())Print  (Q.get ())print(Q.get ())

View CodeLast in, first out, stack queue. Lifoqueue ()

Import Queueq=queue. Lifoqueue (3) q.put (1) q.put (2) q.put (3)print(Q.get ())  Print(q.get ())print(Q.get ())

View CodePriority queue. Priorityqueue ()

Import Queueq # priority, the priority is expressed numerically, the smaller the number, the higher the Priority Q.put ((Ten, 'a')) Q.put ((-1,'b  ')q.put ((+,'C'))print (Q.get ()) Print (Q.get ()) Print (Q.get ())

View Code

Getting started with Python learn-day36-gil global interpreter locks, deadlock phenomena and recursive locks, semaphores, event events, thread queue