Multithreading: Multithreaded design Mode (iii): Master-worker mode

Master-worker mode is one of the common parallel patterns, its core idea is that the system has two processes work together: master process, responsible for receiving and assigning tasks, worker process, responsible for processing subtasks. When the worker process finishes processing the child tasks, the results are returned to the master process, summarized by the master process, and finally the results are obtained.

First, what is the Master-worker mode:

Structure diagram for this pattern:

Structure diagram:

Worker: Used to actually handle a task;

Master: Assignment of tasks and synthesis of the final result;

Main: Start the program and the scheduler opens master.

Second, the code implementation:

The following is a simple implementation of the Master-worker framework.

(1) Master section:

[Java]View PlainCopy

2. Package masterworker;
6. Import Java.util.HashMap;
8. Import Java.util.Map;
10. Import Java.util.Queue;
12. Import Java.util.concurrent.ConcurrentHashMap;
14. Import Java.util.concurrent.ConcurrentLinkedQueue;
18. Public class Master {
22. //Task Queue
24. protected queue<object> workqueue= new concurrentlinkedqueue<object> ();
26. //worker Process Queue
28. protected map<string,thread> threadmap= new hashmap<string,thread> ();
30. //Sub-task processing result set
32. protected map<string,object> resultmap= new concurrenthashmap<string, object> ();
34. //Whether all sub-tasks are over
36. Public Boolean iscomplete () {
38. for (map.entry<string, thread> entry:threadMap.entrySet ()) {
40. if (Entry.getvalue (). GetState ()!=thread.state.terminated) {
42. return false;
44. }
48. }
50. return true;
52. }
56. //master constructs, requires a worker process logic, and the number of worker processes required
58. Public Master (worker worker,int countworker) {
62. Worker.setworkqueue (WorkQueue);
64. Worker.setresultmap (RESULTMAP);
66. For (int i=0;i<countworker;i++) {
68. Threadmap.put (Integer.tostring (i), new Thread (worker, integer.tostring (i)));
70. }
74. }
78. //Submit a task
80. public void Submit (Object job) {
82. Workqueue.add (Job);
84. }
90. //Return child task result set
92. Public map<string,object> Getresultmap () {
94. return resultmap;
96. }
102. //Start running all worker processes for processing
104. public Void execute () {
106. for (map.entry<string, thread> entry:threadMap.entrySet ()) {
108. Entry.getvalue (). Start ();
112. }
114. }
120. }

(2) Worker process implementation:

[Java]View PlainCopy

2. Package masterworker;
6. Import Java.util.Map;
8. Import Java.util.Queue;
12. Public class Worker implements runnable{
16. //Task queue for sub-task acquisition
18. protected queue<object> workQueue;
20. //Sub-task processing result set
22. protected map<string,object> resultmap;
24. public void Setworkqueue (queue<object> workQueue) {
26. this.workqueue= WorkQueue;
28. }
32. public void Setresultmap (map<string,object> resultmap) {
34. This.resultmap=resultmap;
36. }
38. //Sub-task processing logic, implementing specific logic in subclasses
40. Public object handle (object input) {
42. return input;
44. }
50. @Override
52. public Void Run () {
56. While (true) {
58. //Get child tasks
60. Object input= Workqueue.poll ();
62. if (input==null) {
64. Break ;
66. }
68. //Handling subtasks
70. Object re = handle (input);
72. Resultmap.put (Integer.tostring (Input.hashcode ()), re);
74. }
76. }
80. }

(3) Using this small framework to calculate 1--100 's cubic and, Plusworker implementations:

[Java]View PlainCopy

2. Package masterworker;
6. Public class Plusworker extends Worker {
10. @Override
12. Public object handle (object input) {
16. Integer i = (integer) input;
18. return i*i*i;
20. }
26. }

(4) The main function for the calculation:

[Java]View PlainCopy

2. Package masterworker;
6. Import Java.util.Map;
8. Import Java.util.Set;
12. Public class Main {
18. /** 
20. * @param args
22. */
24. public static void Main (string[] args) {
26. //fixed use of 5 worker and specified worker
28. Master m = new Master (new Plusworker (), 5);
30. //Submit 100 sub-tasks
32. For (int i=0;i<100;i++) {
34. M.submit (i);
36. }
38. //Start calculation
40. M.execute ();
42. int re= 0;
44. //Save Final settlement results
46. Map<string,object> Resultmap =m.getresultmap ();
50. //Do not wait for all workers to complete to begin calculating the final result
52. While (Resultmap.size () >0 | |!m.iscomplete ()) {
54. set<string> keys = Resultmap.keyset ();
56. String key =null;
58. For (String k:keys) {
60. Key=k;
62. Break ;
64. }
66. Integer i =null;
68. if (key!=null) {
70. I= (Integer) resultmap.get (key);
72. }
74. if (i!=null) {
76. //FINAL result
78. Re+=i;
80. }
82. if (key!=null) {
84. //Remove items that have already been calculated
86. Resultmap.remove (key);
88. }
92. }
98. }
102. }

Three, Summary:

Master-worker mode is a method of parallelization of serial tasks, the decomposed subtasks can be processed in parallel in the system, and if necessary, the master process does not have to wait for all subtasks to complete the calculation, it can calculate the final result set according to the existing partial result set.

Ext.: http://blog.csdn.net/lmdcszh/article/details/39698189

Multithreading: Multithreaded design Mode (iii): Master-worker mode