Detailed Python multithreaded _python

The example of this article for everyone to analyze Python multithreading, for everyone to refer to, the specific content as follows

1, multithreading of understanding

Multiple processes and multithreading can perform more than one task, and threads are part of the process. Threading is characterized by the sharing of memory and variables between threads, less resource consumption (although in the UNIX environment, multiple processes and multithreaded resource scheduling consumption gap is not obvious, Unix scheduling faster), the disadvantage is that synchronization between threads and lock more cumbersome.

2, Python multithreaded creation

In Python, you can also implement multithreading, with two standard modules thread and threading, but we mainly use more advanced threading modules. Use examples:

Import Threading
Import time
 
def target ():
 print ' curent threading%s is running '% THREADING.CURRENT_THR EAD (). Name
 time.sleep (1)
 print ' The curent threading%s is ended '% Threading.current_thread (). Name
 
print ' The curent threading%s is running '% threading.current_thread (). Name
t = Threading. Thread (Target=target)
 
T.start ()
t.join ()
print ' The curent threading%s is ended '% Threading.current_ Thread (). Name
 

Output:

The curent threading Mainthread is running the curent threading Thread-1 is
running the
curent threading Is ended the
curent threading Mainthread is ended

Start is the start thread, the join is blocking the current thread, even if the current thread ends, it does not exit. As you can see from the result, the main thread does not end until the Thread-1 is finished.

Python, by default, if you do not add a join statement, the main thread does not wait until the current thread finishes, but does not immediately kill it. If you do not add the join output as follows:

The curent threading Mainthread is running the curent threading Thread-1 is
running the
 curent threading D is ended the
curent threading Thread-1 is ended

However, if you add T.setdaemon (True) to the thread instance, if you do not add a join statement, the child thread is killed when the main thread ends. Code:

Import Threading
Import time
def target ():
 print ' curent threading%s is running '% THREADING.CURRENT_THR EAD (). Name
 Time.sleep (4)
 print ' The curent threading%s is ended '% Threading.current_thread (). Name
print ' The curent threading%s is running '% threading.current_thread (). Name
t = Threading. Thread (Target=target)
T.setdaemon (True)
T.start ()
t.join ()
print ' curent threading%s is Ended '% Threading.current_thread (). Name

The output is as follows:

The curent threading Mainthread are running the curent threading Thread-1 is runningthe curent threading Mainthread are
Ended

If you add a join and set the wait time, you wait for the thread to quit for a period of time:

Import Threading
Import time
def target ():
 print ' curent threading%s is running '% THREADING.CURRENT_THR EAD (). Name
 Time.sleep (4)
 print ' The curent threading%s is ended '% Threading.current_thread (). Name
print ' The curent threading%s is running '% threading.current_thread (). Name
t = Threading. Thread (Target=target)
T.setdaemon (True)
T.start ()
t.join (1)

Output:

The curent threading Mainthread is running the curent threading Thread-1 is
running the
curent threading EAD is ended

The main thread waits 1 seconds, ends automatically, and kills the child thread. If the join does not wait, the T.join () waits until the child thread ends, and the output is as follows:

The curent threading Mainthread is running the curent threading Thread-1 is
running the
curent threading 1 is ended the
curent threading Mainthread is ended

3, Thread Lock and threadlocal

(1) Thread lock

For multithreading, the biggest feature is that the thread can share data, then the sharing of data will be multiple threads to change a variable, using the same resources, and the deadlock, data disorder and so on.

Suppose there are two global resources, A and B, with two threads thread1,thread2. Thread1 occupy a, want to access B, but at this time thread2 occupy B, want to access a, two threads do not release the resources at this time, it will cause deadlock.

Lock is present for this issue. When accessing a resource, lock the resource with Lock.acquire (), and after accessing it, release the resource with Lock.release ().

A = 3
lock = Threading. Lock ()
def target ():
 print ' curent threading%s is running '% threading.current_thread (). Name
 Time.sleep (4)
 Global a
 lock.acquire ()
 try:
 a = 3
 finally:
 lock.release ()
 print ' The Curent threading%s is ended '% Threading.current_thread (). Name
 print ' Yes '

The purpose of finally is to prevent the current thread from consuming resources.

(2) ThreadLocal

After introducing the thread lock, the next step is threadlocal. Local variables can be used when you do not want to share variables with other threads, but defining local variables in a function makes it difficult to pass between functions. Threadlocal is a very cool thing, it solves the problem of global variables needing shackles, local variables passing the trouble of two. Defined by thread:

Local_school = Threading.local ()

At this point, the local_school becomes a global variable, but the global variable is only a global variable in that thread, and is a local variable for other threads and cannot be changed by another thread. def process_thread (name): # binding Threadlocal student:local_school.student = Name

This student property can only be modified by this thread and not by another thread. Code:

Local = Threading.local ()
def func (name):
 print ' current thread:%s '% Threading.currentthread (). Name
 Local.name = name
 print '%s in%s '% (Local.name,threading.currentthread (). Name)
T1 = Threading. Thread (target=func,args= (' Haibo ',))
t2 = Threading. Thread (target=func,args= (' Lina ',))
T1.start ()
T2.start ()
t1.join () t2.join ()

As you can see from the code, you can interpret threadlocal as a dict, and you can bind different variables.

The most common use of threadlocal is that each thread handles an HTTP request, which is the principle used in the flask framework, which uses localstack based on Werkzeug.

4, map implementation of multithreading

For the use of multithreading, we often use thread to create, more cumbersome:

Class Mythread (threading. Thread):
 def init (self):
 threading. Thread.init (self)
def run (self):
 lock.acquire ()
 print Threading.currentthread (). GetName (
 ) Lock.release ()
 
def build_worker (num):
 workers = [] for
 T in range (num):
 work = Mythread ()
 Work.start ()
 workers.append (work) return
 workers
def producer ():
 threads = Build_worker (4) For
 W in Threads:
 w.join ()
 print ' Done '

If you want to create more threads, it will be one by one added to the inside, cumbersome operation, code readability is also worse. In Python, you can use the map function to simplify your code. A map can implement multitasking concurrency, simple examples:

Copy Code code as follows:

URLs = [' http://www.baidu.com ', ' http://www.sina.com ', ' http://www.qq.com ']
Results=map (Urllib2.urlopen,urls)

Map passes each element of the URL as a parameter to the Urllib2.urlopen function, and finally puts the result in the results list, and the map function single-handedly a series of operations, such as sequence operation, parameter passing, and result saving. Its principle:

The map function is responsible for splitting threads to different CPUs.

There are two libraries in Python that contain the map function: multiprocessing and its little-known fader Multiprocessing.dummy.dummy is a complete clone of the multiprocessing module, the only difference being that Multip The rocessing acts on the process, while the dummy module acts on the thread. Code:

Import Urllib2
 
from multiprocessing.dummy import Pool as ThreadPool
 
urls = [' http://www.baidu.com ', ' http:// Www.sina.com ', ' http://www.qq.com ']
 
pool = ThreadPool ()
 
results = Pool.map (urllib2.urlopen,urls)
Print Results
pool.close ()
pool.join ()
 
print ' main ended '

pool = ThreadPool () creates a thread pool whose default value is the kernel number of the current machine CPU and can specify the thread pool size, not the more the better, because the more you do, the more the switch between threads is consuming resources.

results = Pool.map (urllib2.urlopen,urls) The statement passes different URLs to the respective threads and returns the results of execution back to the results.

The code is clear and clever to complete the function of the threading module.

5, Python Multi-threading flaw:

It says so much about multithreading, but Python multithreading does not really work because in Python there is a Gil, a global interpretation lock, which guarantees that only one thread at a time can perform a task, that is, multithreading is not real concurrency, but it can be performed alternately. If there are 10 thread cannons on the 10 core CPU, the current work can only be a thread on the CPU.

6. Python Multi-Threading application scenario.

Although Python multithreading is flawed, it is always said to be chicken ribs, but it is not useless, it is suitable for use in IO-intensive tasks. Most I/o-intensive execution time is spent on I/O, such as database I/O and less time spent on CPU computing. So this scenario can use Python multithreading, and when a task is blocked on IO operations, we can immediately switch to execute other IO operation requests on other threads.

Summary: Python multithreading is useful in IO-intensive tasks, but for compute-intensive tasks, you should use Python multiple processes.

The above is the entire content of this article, I hope to help you learn, but also hope that we support the cloud habitat community.