Python multithreading and multi-process (i)

One, multi-threaded

1. Python's standard library provides two modules: _thread and threading , is the _thread low-level module, threading is the Advanced module, the _thread encapsulation. In most cases, you only need to use threading this module.   

   ImportThreading fromTimeImportCtime,sleep
   defFunc1 (func):PrintU"I am the function%s. Printing time:%s"%(Func,ctime ()) Sleep (5)    PrintU'%s Print time%s again'%(Func,ctime ())
   defFunc2 (func):PrintU"I am the function%s now printing time:%s"%(Func,ctime ()) Sleep (5)    PrintU'%s Print time%s again'%(Func,ctime ())
   Threads=[]t1= Threading. Thread (target=func1,args= ('func1',)) #创建线程, the args parameter must be passed into threads.append (T1) in a tuple T2= Threading. Thread (target=func2,args= ('Func2',))#创建线程, the args parameter must be passed in as a tuple
   Threads.append (T2)
   if __name__=='__main__':
    forTinchThreads:
   # T.setdaemon (True) #设置守护线程 (the main thread exits, the child thread exits, and the resource is not suspended)
   T.start () #启动线程
   PrintU'script End Time%s'%ctime ()

   Operation Result: (note Setdaemon ())

   I am the function func1. Printing time: Fri Sep 15:43:15 201815:43:152018 by 15:43:15 2018  # (Three are printed at the same time)
   
   # (two are all waiting for 5 seconds after a colleague prints)

   Func1,func2 are all performing tasks at the same time, but why the script end time is printed first, because the main thread is divided into two sub-threads at the same time, but the child thread has to wait 5 seconds but the main thread does not wait for the direct execution of the print;

2. let's make a little improvement:

   ImportThreading fromTimeImportCtime,sleepdefFunc1 (func):PrintU"I am the function%s. Printing time:%s"%(Func,ctime ()) Sleep (5)    PrintU'%s Print time%s again'%(Func,ctime ())defFunc2 (func):PrintU"I am the function%s now printing time:%s"%(Func,ctime ()) Sleep (5)    PrintU'%s Print time%s again'%(Func,ctime ()) Threads=[]t1= Threading. Thread (target=func1,args= ('func1',)) Threads.append (t1) T2= Threading. Thread (target=func2,args= ('Func2',)) Threads.append (T2)if __name__=='__main__':     forTinchThreads:#T.setdaemon (True) #设置守护线程 (the main thread exits, the child thread exits, and the resource is not suspended)T.start () forTinchThreads:
   T.join ()#wait for thread execution to finish
    Print u' script end time%s'%ctime ()

   I am the function func1. Printing time: Fri Sep 07 16:09:29 2018 I am the function Func2 now printing time: Fri Sep 16:09:29 2018func1 re-print time Fri Sep 16:09:34 2018fun C2 Print Time Fri Sep 07 16:09:34 2018 Script End time Fri Sep 07 16:09:34 2018

   　　The main thread executes to, and then waits for the child thread;

3. The last time I used a thread to write something, I started to write a crawler last year because of a lot of information on the Internet, patchwork finally got me out.

   #Encoding:utf-8ImportRequestsImportThreading fromlxmlImportetreeImportCodecs,csvdefreturn_link (): Link= []     forIinchRange (2000): URL="http://www.xzgsj.gov.cn/baweb/show/shiju/gg.jsp?fenceid=95000000&total=45192&querytype=6& pagenum="+str (i) +"&action=gg.jsp?ssid=ZxSlh1DBkGGpV2hnWMSt5PG7f8FPjTbvH78vJ1nFcwYyMhWzGBzy!1470507879!1500631589968& Findwenhao=&findname="link.append (URL)returnLinkdefget_href (): whileTrue:lock.acquire ()ifLen (Link) = =0:lock.release () Break        Else: URL=link.pop (0) lock.release () R= Requests.get (URL, headers=headers). ContentPrintURL pages= Etree. HTML (R.decode ('Utf-8')) a= Pages.xpath (U"//table[@width = ']/tr[position ' () >1]/td/a/@onclick")             forIincha:a= I.replace ('javaScript:window.open (\ '',"'). Replace ('\ ', \ ' a\ ', \ ' width=550 height=350\ ')',"') URL='http://www.xzgsj.gov.cn/baweb/show/shiju/'+A with Codecs.open ('Url.csv','AB') as F:w=Csv.writer (f) w.writerow ([url])if __name__=='__main__': Headers= {        "user-agent":"mozilla/5.0 (Windows NT 6.1; Win64; x64) applewebkit/537.36 (khtml, like Gecko) chrome/59.0.3071.115 safari/537.36"} lock= Threading. Lock ()#Thread LockLink =Return_link () Tasks= []#Task List     forXinchRange (5): T= Threading. Thread (TARGET=GET_HREF)#preparing thread functions and parametersT.setdaemon (True)#Setting the daemon thread (the main thread exits, the child thread exits, and the resource is not suspended)tasks.append (t) forTinchTasks:t.start ()#Start Multi-Threading (how many threads will start if the task List has a number of values)     forTinchTasks:t.join ()#wait for thread execution to finish

   View Code
4. Use the threading module to create the thread directly from the threading. Thread inherits, and then overrides the __init__ method and the Run method: (as with the first example, just based on the use of a class)

   ImportThreading fromTimeImportCtime,sleepclassMyThread (Threading. Thread):#inherits the parent class threading. Thread    def __init__(self, ThreadID, name, counter): Threading. Thread.__init__(self) self.threadid=ThreadID Self.name=name Self.counter=counterdefRun (self):#write the code you want to execute into the run function. The thread runs the run function directly after it is created        Print "starting"+ Self.name,u'Time {}'. Format (CTime ()) Sleep (5)        Print "Exiting"+ Self.name,u'Time {}'. Format (CTime ())#Create a new threadThread1 = MyThread (1,"Thread-1", 1) Thread2= MyThread (2,"Thread-2", 2)#Open Threadtasks =[]tasks.append (Thread1) tasks.append (thread2)if __name__=='__main__':     forTinchtasks:#T.setdaemon (True) #设置守护线程 (the main thread exits, the child thread exits, and the resource is not suspended)T.start () forTinchTasks:t.join ()#wait for thread execution to finish    PrintU'script End Time%s'%ctime ()
   Results:

   Starting Thread-1 Time Fri Sep 17:13:53 2018starting Thread-2 time Fri Sep 07 17:13:53 2018

   Exiting Thread-1 Time Fri Sep 07 17:13:58 2018
   Exiting Thread-2 time Fri Sep 07 17:13:58 2018

   07 17:13:58 201

    

6. The first one over, and then think about adding it

Python multithreading and multi-process (i)