python\ Processes and Threads 3

1 multiprocessing Module

(1.) Direct import from multiprocessing import processimport osimport timedef info (name):    print ("Name:", name)    print (' Parent process: ', Os.getppid ())    print (' Process ID: ', os.getpid ())    print ("------------------") def foo (name):    info (name)    time.sleep () if __name__ = = ' __main__ ':    info (' main process line ')    p1 = Process (target= info, args= (' Alvin ',))    P2 = Process (Target=foo, args= (' Egon ',)) P1.start () P2.start () p1.join (    )    p2.join ()    print ("Ending") time.sleep (+) >>name:main process lineparent process:16976process ID: 18456------------------name:alvinparent process:18456process id:19884------------------name:egonparent process: 18456process id:19112------------------Ending

　　

(2. ) Methods for creating classes

Construction Method:

Process ([group [, Target [, name [, args [, Kwargs]]])

Group: Thread groups, currently not implemented, the library reference must be none;

Target: The method to be executed;

Name: Process name;

Args/kwargs: The parameter to pass in the method.

Example method:

Is_alive (): Returns whether the process is running.

Join ([timeout]): Blocks the process of the current context until the process calling this method terminates or arrives at the specified timeout (optional parameter).

Start (): Process ready, waiting for CPU scheduling

Run (): Strat () calls the Run method, and if the instance process does not have an incoming target set, this star executes the T default run () method.

Terminate (): Stops the worker process immediately, regardless of whether the task is completed

Property:

Daemon: Same as thread's Setdeamon function

Name: Process name.

PID: Process number.

2 co-process

Advantages of the co-process:

(1) because the single thread does not exist switch

(2) no longer has any concept of lock

Yield is the most basic Ctrip function no way to monitor the IO, the switch can be saved to the state of the data by the Send method to run Import time# Note that the consumer function is a generator (generator): # Any function that contains the yield keyword automatically becomes the generator (generator) object Def consumer ():    r = "While    True:        n = yield R if not        N:            return        print (' [consumer]←←consuming%s ... '% n)        time.sleep (1)        r = ' OK ' def Produce (c):    # 1, First call C.next () to start the generator    next (c)    n = 0 while    N < 5:        n = n + 1        print (' [producer]→→producing%s ... '% n)        # 2, then, once the thing is produced, switch to consumer execution by C.send (n);        cr = C.send (n)        # 4, produce get the result of consumer processing, continue to produce the next message;        print (' [PRODUCER] Consumer return:%s '% cr)    # 5, produce decided not to produce, through C.close () Close Consumer, the entire process is over.    c.close () if __name__== ' __main__ ':    # 6, the entire process is unlocked, executed by one thread, produce and consumer collaborate to complete the task, so called "co-process", rather than the thread's preemptive multitasking.    C = consumer ()    Produce (c)

　　

Greenlet Module

allows for manual switching

Call Property Swich

Gevent can realize IO monitoring

Gevent.joinall Open All Programs

Gevent.spawn switching

3 IO Model

IO refers to input, output

Objects and steps involved in IO occurrence

Two system objects are involved, one is the process (or thread) that calls the IO, and the other is the system kernel (kernel). When an operation occurs, it goes through two stages:

(1) Waiting for data preparation

(2) Copying data from the kernel to the process

IO Model Type:

1. 1. blocking IO

1. 1. non-blocking IO

Non-blocking IO: Sending multiple system calls

Pros: Wait for data is not blocked

Cons: Too many system calls

Can't get the data in time.

Two stages: Wait for data non-blocking

Copy data blocking

After a non-blocking recvform system call is called, the process is not blocked, the kernel is returned to the process immediately, and an error is returned if the data is not ready. After the process returns, it can do something else before initiating the recvform system call. Repeat the above process and iterate through the recvform system calls. This process is often called polling. Poll the kernel data until the data is ready, and then copy the data to the process for data processing. It is important to note that the process of copying data is still a blocking state.

1. 1. IO multiplexing (monitoring multiple links)

Features: (1) Full block

Ability to listen to multiple file descriptors for concurrency

#服务端import selectimport socketsock=socket.socket () #产生一个套接字sock. Bind (("127.0.0.1", 8080)) Sock.listen (5) Sock.setblocking (False) inputs=[sock,]while 1:    r,w,e=select.select (inputs,[],[]) #监听有变化的套接字sock    #wait for Data for    obj in R:        if Obj==sock:            conn,addr=obj.accept () #从内核copy信息到用户态            print ("conn", conn)            Inputs.append (conn) #监听列表添加客户conn        else:            data=obj.recv (1024x768) #接收信息            print (Data.decode ("UTF8"))            Send_data=input (">>") #发送信息            obj.send (Send_data.encode ("UTF8")) #客户端import Socketsock=socket.socket () Sock.connect (("127.0.0.1", 8080)) while 1:    data=input ("input>>")    Sock.send (Data.encode ("UTF8"))    Recv_data=sock.recv (1024x768)    print (Recv_data.decode ("UTF8")) Sock.close ()

　　

For file descriptors (socket objects)

(1) is a non-0 integer that does not change

(2) When sending and receiving data, for the receiving end, the data is first to the kernel space, and then copy to the user space, while the kernel space data is cleared

1. 1. Asynchronous IO

Non-blocking throughout

5. Drive Signal

Summary:

There is a blockage blocking

Non-blocking non-blocking

Call blocking IO will always block the corresponding process knows the operation is complete

non-blocking IO in kernel and when the data is prepared, it returns immediately.

Blocking is a synchronous block: blocking non-blocking IO Multiplexing

Non-blocking is asynchronous blocking: asynchronous IO

4 Selectors module

IO multiplexing Implementation Mechanism

Win:select

Linux:select,poll,epoll

Select Cons: 1. Each call to select will copy all FD (file descriptors) to the kernel space, resulting in reduced efficiency

2. Traverse all FD for data access (the most important issue)

3. Maximum number of connections (1024)

Poll: No limit on maximum number of connections

Epoll:1. The first function creates a epoll handle and copies all of the FD (file descriptors) to the kernel space

Only need to copy once

2. Callback function: A function or a function that triggers when an action is successfully completed

Binds a callback function for all FD, but has data access to trigger the callback function

The callback function places the FD in the list

Import Selectorsimport Socketsock=socket.socket () Sock.bind (("127.0.0.1", 8080)) Sock.listen (5) sock.setblocking ( False) sel=selectors. Defaultselector () #根据具体平台选择最佳IO多路机制def Read (Conn,mask): Try:data=conn.recv (1024x768) print (Data.decode ("UTF8" )) Data2=input (">>") Conn.send (Data2.encode ("UTF8")) except Exception:sel.unregister (conn) d EF Accept (Sock,mask): Sel.register (sock,selectors. event_read,accept) conn,addr=sock.accept () Sel.register (conn,selectors. Event_read,read) Sel.register (sock,selectors. event_read,accept) #注册功能while 1:events=sel.select () for Key,mask in Events:print (key.data) #定义的函数 Prin T (key.fileobj) #socket对象 func=key.data obj=key.fileobj func (obj,mask) Breakimport Socketsock=socket.soc Ket () Sock.connect (("127.0.0.1", 8080)) while 1:data=input ("input>>") Sock.send (Data.encode ("UTF8")) Recv_dat A=SOCK.RECV (1024x768) print (Recv_data.decode ("UTF8")) Sock.close ()

　　

5. Queues

Queues are used in multi-threading, multi-process, to protect data

Queue is a data type

Pros: Thread Safety

Import Queueq=queue. Queue (3) #默认是先进先出q. Put (111) q.put ("Hello") q.put (222) print (Q.get ()) print (Q.get ()) print (Q.get ()) >> 111hello222import Queueq=queue. Queue (3) #默认是先进先出q. Put (111) q.put ("Hello") q.put (222) q.put (223,false) #q =queue. Queue (3) Queues definition can only put 3 values, # #超过限额时, return error message print (Q.get ()) print (Q.get ()) print (Q.get ()) Q.get () #没有数据的时候不会报错, only wait for Q.get (False ) #数据为空, error advanced after the import Queueq=queue. Lifoqueue () q.put (111) q.put (5) q.put (+) print (Q.get ()) Priority import Queueq=queue. Priorityqueue () Q.put ([4, "Hello"]) q.put ([1, "Hello5"]) print (Q.get ())

　　

python\ Processes and Threads 3