Python crawler "2nd article"

One, multi-process

1.fork method (OS module, for LUnix systems)

Fork method: Call 1 times and return 2 times. Cause: The operating system replicates a process (child process) through the current process (the parent process), the two processes are almost identical, the fork method is returned in the parent process, the child process, the child process returns the value 0, and the parent process returns the ID of the child process.

Normal method: Call 1 times, return 1 times

ImportOSif __name__=='__main__':    Print 'Current Process (%s) start ....'% (Os.getpid ())#getpid () user gets the current process IDPID =os.fork ()ifPID <0:Print 'Error in Fork'    elifPID = =0:Print 'I AM Child process (%s)'  andMy parent process is(%s)', (Os.getpid (), Os.getppid ())    Else:        Print 'I (%s) created a child process (%s).', (Os.getpid (), PID) operation results are as follows: Current Process (3052) Start .... I (3052) created a child process (3053). I am Child process (3053) andMy parent process is(3052)

2.multiprocessing (cross-platform)

ImportOS#importing the Process class from the multiprocessing module fromMultiprocessingImportProcessdefRun_proc (name):Print 'Child Process%s (%s) Running ...'%(Name,os.getpid ())if __name__=='__main__':    Print 'Parent process%s.'%os.getpid () forIinchRange (5): P= Process (target = Run_proc,args =(str (i),))Print 'Process would start'        #used to start a processP.start ()#used to implement inter-process synchronizationP.join ()Print 'Process End'The execution results are as follows: Parent process2392. Process would start. Process would start. Process would start. Process would start. Process would start. Child process2 (10748) runing ... Child Process 0 (5324) runing ... Child process1 (3196) runing ... Child process3 (4680) runing ... Child process4 (10696) runing ... Process End

Python crawler "2nd article"