python爬蟲【第2篇】

標籤：end   unix   print   爬蟲   parent   ror   err   tpi   nbsp   

一、多進程

1.fork方法（os模組,適用於Lunix系統）

fork方法：調用1次，返回2次。原因：作業系統經當前進程（父進程）複製出一份進程（子進程），兩個進程幾乎完全相同，fork方法分別在父進程、子進程中返回，子進程傳回值為0，父進程中返回的是子進程的ID。

普通方法：調用1次，返回1次

import osif __name__ == ‘__main__‘:    print ‘current Process (%s) start ....‘%(os.getpid())        #getpid()使用者擷取當前進程ID    pid = os.fork()    if pid <0:        print ‘error in fork‘    elif pid == 0:        print ‘I am child process (%s)‘ and my parent process is (%s)‘,(os.getpid(),os.getppid())    else:        print ‘I (%s) created a child process (%s).‘,(os.getpid(),pid)運行結果如下：current Process (3052) start ....I (3052) created a child process (3053).I am child process (3053) and my parent process is (3052)

2.multiprocessing（跨平台）

import os# 從multiprocessing模組中匯入Process類from multiprocessing import Processdef run_proc(name):    print ‘Child process %s (%s) Running...‘ % (name,os.getpid())if __name__ == ‘__main__‘:    print ‘Parent process %s.‘ % os.getpid()    for i in range(5):        p = Process(target = run_proc,args = (str(i),))        print ‘Process will start‘        #用於啟動進程        p.start()    # 用於實現進程間的同步    p.join()    print ‘Process end‘執行結果如下：Parent process 2392.Process will start.Process will start.Process will start.Process will start.Process will start.Child process 2 (10748) Runing...Child process 0 (5324) Runing...Child process 1 (3196) Runing...Child process 3 (4680) Runing...Child process 4 (10696) Runing...Process end        

 

python爬蟲【第2篇】