Python (8) threads, Processes

Thread

1. what is a thread?

A thread is the smallest unit that the operating system can perform operations On. It is included in the process and is the actual operating unit of the Process. A thread refers to a single sequential control flow in a process in which multiple threads can be concurrent and each thread performs different tasks in Parallel.

2.python GIL Global interpreter lock (only need to Know)

No matter how many threads you start, how many CPUs you have, Python will be calm at the same time to allow only one thread to Run.

The first thing to make clear is that the Gil is not a Python feature, it is a concept introduced when implementing the Python parser (CPython). Just like C + + is a set of language (syntax) standards, but can be compiled into executable code with different Compilers. Well-known compilers such as Gcc,intel c++,visual C + +. Python is the same, and the same piece of code can be executed through different Python execution environments such as Cpython,pypy,psyco. Like the Jpython there is no Gil. however, because CPython is the default Python execution environment for most environments. So in a lot of People's Concept CPython is python, also take for granted the Gil to the Python language flaw. So let's be clear here: Gil is not a Python feature, python can be completely independent of the Gil

This article thoroughly analyzes the Gil's effect on Python multithreading, and it is highly recommended to look at: http://www.dabeaz.com/python/UnderstandingGIL.pdf

3.python Threading Module

The threading module is built on the _thread module. The thread module processes and controls threads in a low-level, primitive way, while the threading module provides a more convenient API to process threads by encapsulating the thread two Times.

There are two ways to call a thread, as Follows:

1) Call directly

Import Threading
Import Time
def Sayhi (num): #定义每个线程要运行的函数
Print ("running on Number:%s"%num)
Time.sleep (3)
if __name__ = = ' __main__ ':
T1 = Threading. Thread (target=sayhi,args= (1,)) #生成一个线程实例 target= function name args to the tuple, which is the parameter in the tuple
T2 = Threading. Thread (target=sayhi,args= (2,)) #生成另一个线程实例
T1.start () #启动线程
T2.start () #启动另一个线程
Print (t1.getname ()) #获取线程名
Print (t2.getname ())

2) inherited calls

Class MyThreaddef run

Python provides support for threads through the two standard library thread and Threading. The thread provides a low-level, primitive thread, and a simple lock.

Thread additional methods provided by the Module:

• Threading.currentthread (): returns the current thread Variable.
• Threading.enumerate (): Returns a list that contains the running Thread. Running refers to threads that do not include Pre-and post-termination threads until after the thread has started and ENDS.
• Threading.activecount (): Returns the number of running threads with the same result as Len (threading.enumerate ()).

In addition to using methods, the thread module also provides the thread class to handle threads, and the thread class provides the following methods:

• Run (): the method used to represent thread Activity.
• Start (): initiates thread Activity.
• Join ([time]): waits until the thread aborts. This blocks the calling thread until the Thread's join () method is called abort-gracefully exits or throws an unhandled exception-or an optional timeout occurs.
• IsAlive (): Returns whether the thread is Active.
• GetName (): returns the thread Name.
• SetName (): Sets the thread Name.

4.Join & Daemon

Join after waiting for the thread to finish executing, the other threads continue to execute

def run__name__ = = ' __main__ ': t1 = threading. Thread (target=run,args= ("t1", 2)) t2 = threading.     Thread (target=run,args= ("t2", 3)) # Two executes at the same time, and then waits for T1 execution to finish, and then the main thread and child threads start executing t1.start () t2.start () t1.join () # wait for T1 Print ("main Thread") # program output # test ... t1 # test ... t2 # test...done T1 # main thread # Test...done T2

Daemon Daemon Process

T.setdaemon () is set to either a background thread or a foreground thread (default: False), and a Boolean value that sets whether the thread is a daemon thread and must be used after the start () method is Executed. If it is a background thread, during the main thread execution, the background thread is also in progress, after the main thread executes, the background thread will stop whether or not it succeeds or not; if the foreground thread is executing, the foreground thread is also in progress, and after the main thread executes, waiting for the foreground thread to finish executing, the program stops

ImportThreading,timedef Run(n): print (' [%s]------running----\ n '% n) time.sleep (2) Print ('--done--')def main(): forIinchRange (5): T = threading.  Thread (target=run, args=[i,]) t.start () t.join (1) print (' starting thread ', t.getname ()) m = Threading. Thread (target=main, Args=[]) M.setdaemon (True# Sets the main thread as the daemon thread, which is the daemon thread of the Program's main threads, and the # m thread exits when the main path exits, and other sub-threads initiated by M will exit at the same time, regardless of whether the task is performed M.start () M.join (timeout=2) Print ("---main thread Done----") # program output # [0]------running----# starting thread Thread-2 # [1]------r Unning----#--done--#---main Thread done----

5. thread lock (mutex Mutex)

When we use threads to manipulate data, if multiple threads modify a data at the same time, unexpected results can occur, and the concept of locking is introduced to ensure the accuracy of the Data.

Example: assuming that all elements of List A is 0, when one thread prints all the elements of the list backwards from the previous one, and the other thread modifies the List's elements from the back to 1, then the elements of the list will be partially 0 and some 1 in the output, resulting in inconsistent Data. The presence of the lock solves the Problem.

No lock:

defaddnumthread_list:# Wait for all threads to execute t.join () print (' final num: ', Nu M

Locking

defaddnumthread_list:# Wait for all threads to execute t.join () print (' final num: ', Nu M

GIL VS LOCK

The witty classmate may ask this question, that is, since you said before, Python already has a Gil to ensure that only one thread can execute at the same time, why do we need lock here? notice, The lock here is the User-level lock, and that Gil does not matter, specifically, we look at the + with my live talk to everyone, i understand.

6. Recursive lock

It's a big lock, and it's going to contain a sub-lock.

ImportThreading,timedef run1(): Print ("grab The first part Data") Lock.acquire ()Globalnum num + = 1 lock.release ()returnNumdef run2(): Print ("grab The second part Data") Lock.acquire ()Globalnum2 num2 + = 1 lock.release ()returnNum2def RUN3(): lock.acquire () res = run1 () print ('--------between run1 and run2-----') res2 = run2 () lock.release () Print (res, Res2)if__name__ = = ' __main__ ': num, num2 = 0, 0 lock = threading. Rlock () forIinchRange (ten): T = threading. Thread (target=run3) T.start () whileThreading.active_count ()! = 1:print (threading.active_count ())Else: Print ('----all threads do---') Print (num, Num2)

Threading. Rlock and threading. The difference between Lock:

Rlock is allowed to be acquire multiple times in the same thread. But lock does not allow this Situation. If you use rlock, then acquire and release must appear in pairs, that is, call n times acquire, must call the N-time release to really release the occupied locks.

Threading lock = Threading. Lock () #Lock对象 lock.acquire () lock.acquire () #产生了死琐. Lock.release () lock.release ()

Threading Rlock = Threading. Rlock () #RLock对象 rlock.acquire () rlock.acquire () #在同一线程内, The program does not clog. Rlock.release () rlock.release ()

1. multi-process multiprocessing

The multiprocessing package is a multi-process Management Pack in Python and is a cross-platform version of a multi-process module. With Threading. Like thread, it can take advantage of multiprocessing. Processes object to create a process. The process can run functions written inside the Python Program. The process object is similar to the usage of the thread Object.

Create a process instance that can be started using the start () method.

The join () method can wait for the child process to end before continuing to run, typically for inter-process Synchronization.

from multiprocessing import Process
Import Time
def F (name):
Time.sleep (2)
Print (' Hello ', name)
if __name__ = = ' __main__ ':
p = Process (target=f, args= (' Bob ',))
P.start ()
P.join ()

Write a program that compares the ID of the main process and the child process:

def info def F      __name__ = = ' __main__ ': info (' \033[32;1mmain process line\033[0m ') p = Process (target=f, args= (' Bob ',)) P.start ()

2. inter- Process Communication

Memory between different processes is not shared, and to achieve data exchange between two processes, you can use queue, Pipe, and Manager, where:

1) Queue \ Pipe only realizes the transfer of data between processes;

2) The Manager realizes the sharing of data between processes, that is, multiple processes can modify the same data;

2.1 Queue

The queue allows multiple processes to be put in, with multiple processes fetching objects from the queue, fifo. (using the same method as the queue in Threading)

def F     (qq): qq.put ([all,None, "hello"]) qq.put ([up,None__name__ = = ' __main__ ': q = Queue () p = Process (target=f,args= (q,)) p.start () print (q.get ()) print (q.get ()) p.join ()

2.2 Pipe

Pipe is also first-out

def F None None  __name__ = = ' __main__ ': parent_conn, child_conn = Pipe () p = Process (target=f, args= (CHILD_CONN)) P. Start () print (parent_conn.recv ()) # prints "[all, none, ' Hello ']" print (parent_conn.recv ()) # prints "[none, ' H Ello ' Parent_conn.send ("home for Dinner!") ") # prints" [all, None, ' Hello '] "p.join ()

2.3 Manager

The

manager object is similar to the communication between the server and the customer (server-client), similar to our activity on the Internet. We use a process as a server to build the manager to really store the Resources. Other processes can be passed through parameters or access the manager based on the address, and after establishing the connection, manipulate the resources on the Server. With the firewall permitting, we can fully apply the manager to multiple computers, mimicking a real-world network scenario.

deffp_list:#等待结果 res.join()

3. Process Pool

The process pool can create multiple Processes. These processes are like soldiers on standby, ready to perform Tasks (procedures). A process pool can accommodate multiple soldiers on Standby.

There are two methods of process pooling:

1) Serial: Apply

2) Parallel: Apply_async

def FoodefBarrange: Pool.apply_async (func=foo, args= (i,), callback=bar) print (' end ') pool.close () pool.join () #进程池中进程执行完毕后在关闭;

The purpose of using the callback function is to improve efficiency in the parent process, such as connecting the database, writing the callback function, the parent process connects to the database once, and if the child process is used, multiple connections are required)

4. other (lock)

Lock: a lock printed on the screen to prevent confusion in printing

def F Try finally  range (ten): Process (target=f, args= (lock, num)). start   ()

Python (8) threads, Processes