Python Development thread: Thread & Daemon Thread & Global Interpreter Lock

Introduction of a threading module

The Multiprocess module completely imitates the interface of the threading module, which has a great similarity in the use level, so it is no longer described in detail

Website Link: https://docs.python.org/3/library/threading.html?highlight=threading#

Two ways to open a thread

#方式一from Threading Import Threadimport timedef Sayhi (name):    time.sleep (2)    print ('%s say hello '%name) if __name_ _ = = ' __main__ ':    t=thread (target=sayhi,args= (' Egon '))    T.start ()    print (' main thread ')

Way One

#方式二from Threading Import Threadimport timeclass Sayhi (Thread):    def __init__ (self,name):        super (). __INIT__ ()        self.name=name    def run (self):        time.sleep (2)        print ('%s say hello '% self.name) if __name__ = = ' __main__ ':    t = sayhi (' Egon ')    T.start ()    print (' main thread ')

Mode two

Three the difference between opening multiple threads under one process and opening multiple sub-processes under one process

From threading import threadfrom multiprocessing import processimport osdef work ():    print (' hello ') if __name__ = = ' __ Main__ ':    #在主进程下开启线程    t=thread (target=work)    t.start ()    print (' main thread/main process ') '    prints the    result:    Hello    main thread/main process    '    #在主进程下开启子进程 t=process (    target=work)    t.start ()    print (' main thread/ Main process ') '    print result: main thread    /main process    Hello    ' '

1 who has the fastest opening speed

From threading import threadfrom multiprocessing import processimport osdef work ():    print (' Hello ', os.getpid ()) if __ name__ = = ' __main__ ':    #part1: Open multiple threads under the main process, each thread is the same as the PID of the main process    T1=thread (target=work)    T2=thread (target= Work)    T1.start ()    t2.start ()    print (' main thread/main process pid ', Os.getpid ())    #part2: Multiple processes are open, each process has a different PID    p1= Process (target=work)    p2=process (target=work)    P1.start ()    p2.start ()    print (' main thread/main process PID ', Os.getpid ())

2 A look at the PID

From  Threading Import threadfrom multiprocessing import processimport osdef work ():    global n    n=0if __name__ = = ' __main__ ':    # n=100    # p=process (target=work)    # P.start ()    # p.join ()    # Print (' main ', N) # There is no doubt that the child process P has changed its global N to 0, but only to its own, to view the parent process n is still    n=1    t=thread (target=work)    T.start ()    t.join ()    print (' primary ', N) #查看结果为0 because the in-process data is shared among threads in the same process

3 threads in the same process share the process's data? Four exercises

Practice One:

#_ *_coding:utf-8_*_#!/usr/bin/env pythonimport multiprocessingimport threadingimport sockets=socket.socket (socket. Af_inet,socket. Sock_stream) S.bind ((' 127.0.0.1 ', 8080)) S.listen (5) def action (conn): While    True:        data=conn.recv (1024x768)        Print (data)        Conn.send (Data.upper ()) If __name__ = = ' __main__ ': While    True:        conn,addr=s.accept ()        p= Threading. Thread (target=action,args= (conn,))        P.start ()

multi-threaded concurrent Socket server

#_ *_coding:utf-8_*_#!/usr/bin/env pythonimport sockets=socket.socket (socket.af_inet,socket. Sock_stream) S.connect ((' 127.0.0.1 ', 8080)) while True:    msg=input (' >>: '). Strip ()    if not msg:continue    s.send (Msg.encode (' Utf-8 '))    data=s.recv (1024x768)    print (data)

Client

Practice 2:3 tasks, one to receive user input, one to format user input into uppercase, and one to save formatted results to a file

From threading import Threadmsg_l=[]format_l=[]def talk ():    while True:        msg=input (' >>: '). Strip ()        If not msg:continue        msg_l.append (msg) def format_msg (): While    True:        if msg_l:            res=msg_l.pop ()            Format_l.append (Res.upper ()) def Save ():    while True:        if format_l:            with open (' Db.txt ', ' a ', encoding= ' Utf-8 ') as F:                Res=format_l.pop ()                f.write ('%s\n '%res) if __name__ = = ' __main__ ':    t1=thread (Target=talk )    T2=thread (target=format_msg)    t3=thread (target=save)    T1.start ()    T2.start () T3.start ( )

View CodeFive other thread-related methods

Thread instance object's method  # isAlive (): Returns whether the thread is active.  # getName (): Returns the thread name.  # SetName (): Sets the thread name. Some of the methods provided by the threading module are:  # threading.currentthread (): Returns the current thread variable.  # threading.enumerate (): Returns a list that contains the running thread. Running refers to threads that do not include pre-and post-termination threads until after the thread has started and ends.  # Threading.activecount (): Returns the number of running threads with the same result as Len (Threading.enumerate ()).

From threading import threadimport threadingfrom multiprocessing import processimport osdef work ():    Import    time Time.sleep (3)    print (Threading.current_thread (). GetName ()) If __name__ = = ' __main__ ':    #在主进程下开启线程    t= Thread (target=work)    t.start ()    print (Threading.current_thread (). GetName ())    print (Threading.current_ Thread ()) #主线程    print (Threading.enumerate ()) #连同主线程在内有两个运行的线程    print (Threading.active_count ())    print ( ' main thread/main process ')    '    print result:    mainthread    <_mainthread (Mainthread, started 140735268892672) >    [<_mainthread (Mainthread, started 140735268892672), <thread (Thread-1, started 123145307557888);] Main thread    /master process    Thread-1 ""    

View Code

The main thread waits for the child thread to end

From threading Import Threadimport timedef Sayhi (name):    time.sleep (2)    print ('%s say hello '%name) if __name__ = = ' __main__ ':    t=thread (target=sayhi,args= (' Egon '))    T.start ()    t.join ()    print (' main thread ')    print ( T.is_alive ())    '    Egon say hello main thread    False    '

Six Daemon threads

Whether it is a process or a thread, follow: Guardian xxx will wait for the main xxx to be destroyed after the completion of the operation

It should be emphasized that the operation is not terminated

#1. For the main process, run complete means that the main process code has finished running. To the main thread, it means that all the non-daemons in the process of the main thread are running, and the main thread is running.

Detailed Explanation:

#1 The main process has finished running after its code is finished (the daemon is being reclaimed at this point), then the main process will always wait until the non-daemon has finished running and reclaim the child process's resources (otherwise it will produce a zombie process) before it ends, #2 The main thread runs after the other non-daemon threads have finished running (the daemon is recycled at this point). Because the end of the main thread means the end of the process, the resources of the process as a whole are recycled, and the process must ensure that the non-daemon threads are finished before they end.

From threading Import Threadimport timedef Sayhi (name):    time.sleep (2)    print ('%s say hello '%name) if __name__ = = ' __main__ ':    t=thread (target=sayhi,args= (' Egon ',))    T.setdaemon (True) #必须在t. Start () before setting    T.start ()    Print (' main thread ')    print (T.is_alive ())    "main thread    True    "

From threading import Threadimport timedef foo ():    print (123)    Time.sleep (1)    print ("end123") def Bar ():    print (456)    Time.sleep (3)    print ("end456") T1=thread (Target=foo) t2=thread (Target=bar) t1.daemon= Truet1.start () T2.start () print ("Main-------")

examples of confusing people

Seven Global interpreter lock Gil Introduction

Definition: In CPython, the global interpreter lock, or GIL, was a mutex that prevents multiple native threads from executing Pyt Hon Bytecodes at once. This lock is necessary mainly because CPython ' s memory management are not thread-safe. (However, since the GIL exists, other features has grown to depend on the guarantees that it enforces.) Conclusion: In the CPython interpreter, multiple threads that are opened under the same process can only have one thread at a time and cannot take advantage of multicore advantages

The first thing to be clear is GIL not the Python feature, which is a concept introduced when implementing the Python parser (CPython). Just like C + + is a set of language (syntax) standards, but can be compiled into executable code with different compilers. Well-known compilers such as Gcc,intel c++,visual C + +. Python is the same, and the same piece of code can be executed through different Python execution environments such as Cpython,pypy,psyco. Like the Jpython there is no Gil. However, because CPython is the default Python execution environment for most environments. So in many people's concept CPython is Python, also take it for granted that the GIL Python language defects. So let's be clear here: Gil is not a python feature, Python can be completely independent of the Gil

This article thoroughly analyzes the Gil's effect on Python multithreading, and it is highly recommended to look at: http://www.dabeaz.com/python/UnderstandingGIL.pdf

Back to the top of 7.1 Gil Introduction

Gil is the essence of a mutex, since it is a mutex, all the nature of the mutex is the same, all the concurrent operation into serial, in order to control the same time shared data can only be modified by a task, and thus ensure data security.

One thing is certain: to protect the security of different data, you should add a different lock.

To understand the Gil, first make a point: each time you execute a python program, you create a separate process. For example, Python Test.py,python Aaa.py,python bbb.py will produce 3 different Python processes

"' #验证python test.py will only produce a process #test.py content import os,timeprint (Os.getpid ()) Time.sleep (+)" Python3 test.py # Under Windows tasklist |findstr python# under Linux ps aux |grep python

verifying that Python test.py only produces a single process

In a python process, not only the main thread of the test.py or other threads opened by the thread, but also the interpreter-level thread of the interpreter-enabled garbage collection, in short, all threads are running within this process, without a doubt

#1 all the data is shared, which, The code as a data is also shared by all threads (all code of test.py and all code of the CPython interpreter) For example: test.py defines a function work (code content), and all threads in the process have access to the code, so we can open three threads and target will point to The code, which can be accessed to mean that it can be executed. #2 the task of all threads requires that the code of the task be passed as a parameter to the interpreter's code to execute, that is, all threads want to run their own tasks, the first thing to do is to have access to the interpreter's code.

Comprehensive:

If multiple threads are target=work, then the execution process is

Multiple lines enters upgradeable access to the interpreter's code, that is, get execute permission, and then give the target code to the interpreter code to execute

The code of the interpreter is shared by all threads, so the garbage collection thread can also access the interpreter's code to execute, which leads to a problem: for the same data 100, it is possible that thread 1 executes the x=100 while garbage collection performs the recovery of 100 operations, there is no clever way to solve this problem , is to lock processing, such as Gil, to ensure that the Python interpreter can only execute one task at a time code

7.2 Gil and lock

The Gil protects the data at the interpreter level and protects the user's own data by locking them up, such as

7.3 Gil and multithreading

With Gil's presence, at the same moment only one thread in the same process is executed

Heard here, some students immediately questioned: The process can take advantage of multicore, but the overhead, and Python's multithreaded overhead, but can not take advantage of multicore advantage, that is, Python is useless, PHP is the most awesome language?

Don't worry, Mom's not finished yet.

To solve this problem, we need to agree on several points:

#1. Is the CPU used for computing, or is it used for I/O? #2. Multi-CPU means that multiple cores can be computed in parallel, so multi-core boosts compute performance. Once each CPU encounters I/O blocking, it still needs to wait, so it's useless to check I/O operations more

A worker is equivalent to the CPU, at this time the calculation is equivalent to workers in the work, I/O blocking is equivalent to work for workers to provide the necessary raw materials, workers work in the process if there is no raw materials, the workers need to work to stop the process until the arrival of raw materials.

If your factory is doing most of the tasks of preparing raw materials (I/O intensive), then you have more workers, the meaning is not enough, it is not as much as a person, in the course of materials to let workers to do other work,

Conversely, if your plant has a full range of raw materials, the more workers it is, the more efficient it is.

Conclusion:

For computing, the more CPU, the better, but for I/O, no more CPU is useless

Of course, to run a program, with the increase in CPU performance will certainly be improved (regardless of the increase in size, there is always improved), this is because a program is basically not pure computing or pure I/O, so we can only compare to see whether a program is computationally intensive or I/o-intensive, Further analysis of the Python multithreading in the end there is no useful

#分析: We have four tasks to deal with, the processing method must be to play a concurrency effect, the solution can be: Scenario One: Open four process Scenario two: a process, open four threads # single core case, the analysis results: If four tasks are computationally intensive, no multicore to parallel computing, Scenario one increases the cost of creating a process, and the scheme wins if four tasks are I/O intensive, the cost of the scenario one creation process is large, and the process is much less than the thread, and the scenario is faster than the threads, the result of the analysis: if four tasks are computationally intensive, multicore means parallel computing, In Python a process in the same time only one thread execution is not multi-core, scheme one win if four tasks are I/O intensive, no more cores can not solve the I/O problem, the scheme wins #结论: Now the computer is basically multicore, Python's efficiency in multi-threading for computationally intensive tasks does not bring much performance gains, or even better serial (no large switching), but there is a significant increase in the efficiency of IO-intensive tasks.

Python Development thread: Thread & Daemon Thread & Global Interpreter Lock