Java concurrent topic: Execute CompletionService ExecutorService. invokeAll for Batch Tasks with returned results

Reprinted please indicate the source: http://blog.csdn.net/lmj623565791/article/details/27250059

In general, we use Runnable as the basic task representation, but Runnable is a very limited abstraction. In the run method, only logs can be recorded and printed, or aggregate data into a container (on the one hand, the memory consumption is high, on the other hand, the synchronization needs to be controlled, and the efficiency is very high). In short, the execution results cannot be returned; for example, if 1000 tasks capture data on the network at the same time, and then process the captured data (the processing method is not fixed), I think the best way is to provide a callback interface, the processing method is the most callback passed in; but now we have a better implementation: CompletionService + Callable

The call method of Callable can return the execution result;

CompletionService combines Executor (thread pool) and BlockingQueue (blocking Queue), and uses Callable as the basic unit of the task. The whole process is that the producer keeps putting the Callable task into blocking, as a consumer, Executor constantly retrieves and executes tasks and returns results;

Advantages:

A. Blocking queues prevent too many tasks waiting in the queue in the memory, resulting in memory overflow (after all, generally, the producer speed is relatively fast. For example, when crawlers prepare websites and rules, they will execute them, execution (consumer) is still slow)

B. CompletionService can be implemented. If a task is executed first, it will be returned instead of returned in order, which greatly improves the efficiency;

1. CompletionService: Executor + BlockingQueue

The following is an example:

Package com. zhy. concurrency. completionService; import java. util. random; import java. util. concurrent. blockingQueue; import java. util. concurrent. callable; import java. util. concurrent. completionService; import java. util. concurrent. executionException; import java. util. concurrent. executorCompletionService; import java. util. concurrent. executorService; import java. util. concurrent. executors; import java. util. concurrent. future; import java. util. concurrent. linkedBlockingDeque;/*** combines Executor and BlockingQueue functions. You can submit the Callable task to it for execution, and then use take () method to obtain the completed result ** @ author zhy **/public class CompletionServiceDemo {public static void main (String [] args) throws InterruptedException, executionException {/*** internally maintains the thread pool of 11 threads */ExecutorService exec = Executors. newFixedThreadPool (11);/*** blocking queue with a capacity of 10 */final BlockingQueue <Future <Integer> queue = new LinkedBlockingDeque <Future <Integer> (10 ); // instantiate CompletionServicefinal CompletionService <Integer> completionService = new ExecutorCompletionService <Integer> (exec, queue);/*** simulate 10 tasks in an instant, and the execution time of each task is inconsistent */for (int I = 0; I <10; I ++) {completionService. submit (new Callable <Integer> () {@ Overridepublic Integer call () throws Exception {int ran = new Random (). nextInt (1000); Thread. sleep (ran); System. out. println (Thread. currentThread (). getName () + "Resting" + ran); return ran ;}};}/*** output result immediately */for (int I = 0; I <10; I ++) {try {// who is the first to complete the execution and returns Future <Integer> f = completionService. take (); System. out. println (f. get ();} catch (InterruptedException e) {e. printStackTrace ();} catch (ExecutionException e) {e. printStackTrace () ;}} exec. shutdown ();}}

Output result:

Pool-1-thread-4 break 5252pool-1-thread-1 break 5959pool-1-thread-10 break 215pool-1-thread-9 Break break 794794pool-1-thread-7 Break break 987987

The direct return after the first execution is completed does not need to be executed in the order submitted by the task. If you need to write a high-concurrency program and each task needs to return the execution result, this is a very good choice!

2. ExecutorService. invokeAll

The invokeAll method of ExecutorService can also execute tasks in batches and return results in batches. However, I think it is a fatal disadvantage. It must be returned after all tasks are completed, on the one hand, the memory is held for a long time; on the other hand, the responsiveness also has a certain impact. After all, you all like to look at the output of the execution result of the brush, rather than waiting for it;

The following is an example:

Package com.zhy.concurrency.exe cutors; import java. util. arrayList; import java. util. list; import java. util. random; import java. util. concurrent. callable; import java. util. concurrent. executionException; import java. util. concurrent. executorService; import java. util. concurrent. executors; import java. util. concurrent. future; public class TestInvokeAll {public static void main (String [] args) throws InterruptedException, ExecutionException {ExecutorService exec = Executors. newFixedThreadPool (10); List <Callable <Integer> tasks = new ArrayList <Callable <Integer> (); Callable <Integer> task = null; for (int I = 0; I <10; I ++) {task = new Callable <Integer> () {@ Overridepublic Integer call () throws Exception {int ran = new Random (). nextInt (1000); Thread. sleep (ran); System. out. println (Thread. currentThread (). getName () + "Resting" + ran); return ran ;}}; tasks. add (task);} long s = System. currentTimeMillis (); List <Future <Integer> results = exec. invokeAll (tasks); System. out. println ("execution task consumption:" + (System. currentTimeMillis ()-s) + "millisecond"); for (int I = 0; I <results. size (); I ++) {try {System. out. println (results. get (I ). get ();} catch (Exception e) {e. printStackTrace () ;}} exec. shutdown ();}}

Execution result:

Pool-1-thread-10, 1 pool-1-thread-5, 59pool-1-thread-6, 128pool-1-thread-1, 3 pool-1-thread-1, 3 pool-1-thread-6, 3 pool-1-thread-2, and 707pool-1-thread-2. now: 819 Ms 146817158707591283876064861

I specifically printed a time after the task is submitted and then printed the next time after the invokeAll execution is completed. It can be seen that the return of invokeAll is waiting for all threads to complete the execution. In this case, I think the availability is not as good as the CompletionService.

Well, this is the case where tasks are executed in batches and returned results are carried ~ If you have any questions or errors in the code, please point out ~