Concurrency model (2) -- Master-Worker Mode

Concurrency model (2) -- Master-Worker Mode
Zookeeper

The Master-Worker mode is one of the commonly used parallel modes. Its core idea is that the system has two processes working collaboratively: the Master process, which is responsible for receiving and allocating tasks; the Worker process, handles subtasks. After the Worker process finishes processing the sub-task, the result is returned to the Master process, which summarizes the Master process and finally obtains the final result.

1. What is the Master-Worker mode:

Structure of this mode:

Structure:

Worker: used to process a task;

Master: assignment of tasks and merging of final results;

Main: start the program and enable the Master for scheduling.

Ii. Code implementation:

The following is a simple Master-Worker Framework implementation.

(1) Master part:

Package MasterWorker; import java. util. hashMap; import java. util. map; import java. util. queue; import java. util. concurrent. concurrentHashMap; import java. util. concurrent. concurrentLinkedQueue; public class Master {// Job Queue protected QueueWorkQueue = new concurrent1_queue(); // Worker Process queue protected Map
   
    
ThreadMap = new HashMap
    
     
(); // The subtask processing result set protected Map
     
      
ResultMap = new ConcurrentHashMap
      
        (); // Whether all subtasks have ended public boolean isComplete () {for (Map. Entry
       
         Entry: threadMap. entrySet () {if (entry. getValue (). getState ()! = Thread. state. TERMINATED) {return false ;}} return true;} // The construction of the Master, requires a Worker process logic, and requires the number of Worker processes public Master (Worker worker, int countWorker) {worker. setWorkQueue (workQueue); worker. setResultMap (resultMap); for (int I = 0; I
        
          GetResultMap () {return resultMap;} // start running all Worker processes and process public void execute () {for (Map. Entry
         
           Entry: threadMap. entrySet () {entry. getValue (). start ();}}}
          
(2) Worker Process implementation:
          

          
Package MasterWorker; import java. util. Map; import java. util. Queue; public class Worker implements Runnable {// task Queue, used to obtain the subtask protected QueueWorkQueue; // The subtask processing result set protected Map
            
             
ResultMap; public void setWorkQueue (Queue
             WorkQueue) {this. workQueue = workQueue;} public void setResultMap (Map
              
               
ResultMap) {this. resultMap = resultMap;} // sub-task processing logic. In the subclass, implement the specific logic public Object handle (Object input) {return input ;}@ Overridepublic void run () {while (true) {// obtain the subtask Object input = workQueue. poll (); if (input = null) {break;} // process the subtask Object re = handle (input); resultMap. put (Integer. toString (input. hashCode (), re );}}}
               

               

               

               

               
(3) Use this small framework to calculate the 1--100 cubic sum and PlusWorker implementation:
               

               
package MasterWorker;public class PlusWorker extends Worker {@Overridepublic Object handle(Object input) {Integer i =(Integer)input;return i*i*i;}}
               

               

               

               

               
(4) Main Function for calculation:
               

               
Package MasterWorker; import java. util. map; import java. util. set; public class Main {/*** @ param args */public static void main (String [] args) {// fixed use of five workers, and specify WorkerMaster m = new Master (new PlusWorker (), 5); // submit 100 subtasks for (int I = 0; I <100; I ++) {m. submit (I) ;}// calculate m.exe cute (); int re = 0; // Save the final settlement result Map
                
                 
ResultMap = m. getResultMap (); // you can start calculating the final result while (resultMap. size ()> 0 without waiting for all workers to complete. |! M. isComplete () {Set
                 
                  
Keys = resultMap. keySet (); String key = null; for (String k: keys) {key = k; break;} Integer I = null; if (key! = Null) {I = (Integer) resultMap. get (key) ;}if (I! = Null) {// The final result re + = I;} if (key! = Null) {// remove the calculated resultMap. remove (key );}}}}
                 
                
               

               

               

               

               
Iii. Summary:
               
The Master-Worker mode is a scheme for parallel processing of serial tasks. The decomposed sub-tasks can be processed in parallel in the system, the Master process can calculate the final result set based on some existing result sets without waiting for all subtasks to complete the calculation.