Python Simple distributed Demo

A server is a master,b server for worker,

Execute taskmanger.py on Server A

#Coding:utf-8ImportRandom,time,queue fromMultiprocessing.managersImportBasemanager#implementing the first step: establishing Task_queue and result_queue for storing tasks and resultsTask_queue=queue.queue () result_queue=Queue.queue ()classQueueManager (basemanager):Pass#Implement the second step: Register the created two queues on the network, using the Register method, the callable parameter is associated with the queue object,#exposing a Queue object to the networkQueuemanager.register ('Get_task_queue', callable=Lambda: Task_queue) Queuemanager.register ('Get_result_queue', callable=Lambda: Result_queue)#Implement the third step: Bind Port 8001, set the authentication password ' Qiye '. This corresponds to the initialization of the objectManager=queuemanager (address= ("', 8001), authkey='LSF')#Implement the fourth step: Start the management, monitor the information channelManager.start ()#Implement step Fifth: Get a Queue object accessed over the network by managing the instance's methodstask=manager.get_task_queue () result=Manager.get_result_queue ()#Implementing step Sixth: adding Tasks forUrlinch["Imageurl_"+STR (i) forIinchRange (10)]:    Print 'put task%s ...'%url task.put (URL)#Get return resultsPrint 'try get result ...' forIinchRange (10):    Print 'result is%s'%result.get (timeout=10)#Close ManagementManager.shutdown ()

Script for worker execution on server B taskworker.py

#Coding:utf-8Import Time fromMultiprocessing.managersImportBasemanager#Create a similar queuemanager:classQueueManager (basemanager):Pass#implementing the first step: Get the method name of the queue using QueueManager registrationQueuemanager.register ('Get_task_queue') Queuemanager.register ('Get_result_queue')#implementing the second step: connecting to the server:SERVER_ADDR ='127.0.0.1'Print('Connect to server%s ...'%server_addr)#the port and authentication password are kept in full alignment with the service process settings:m = QueueManager (address= (SERVER_ADDR, 8001), authkey='LSF')#from the network connection:M.connect ()#implement step three: Get the object for the queue:Task =m.get_task_queue () result=M.get_result_queue ()#Implement step Fourth: Take the task from the task queue and write the results to the result queue: while( notTask.empty ()): Image_url= Task.get (true,timeout=5)        Print('Run task download%s ...'%image_url) Time.sleep (1) Result.put ('%s--->success'%Image_url)#processing Ends:Print('worker exit.')

Python Simple distributed Demo