Java concurrency Programming (iv) blocking queues and producer-consumer patterns

blocking queues

The blocking queue provides a blocking put and take method, as well as an offer and poll method that supports timing. If the queue is full, the put method blocks until there is space available, and if the queue is empty, the take method blocks until an element is available. Queues can be bounded or unbounded, and unbounded queues will never be filled, so put methods on unbounded queues are never blocked. A common blocking producer-consumer pattern is the combination of a thread pool and a work queue, which is reflected in the Executor task execution framework.

meaning : This pattern simplifies the development process because he eliminates code dependencies between producer and consumer classes, and it decouples the process of producing data from the process of using that data to simplify workload management. For I/O-intensive and CPU-intensive producers and consumers, many performance benefits can be brought.

Blocking queues simplifies the coding of consumer programs because the take operation blocks until there is data available. In some cases, this approach is very appropriate (for example, in a server application, waiting without a client request), and to achieve higher resource utilization when there is endless work to be done, such as a web crawler.

The class library contains several implementations of Blockingqueue, where Linkedblockingqueue and Arrayblockingqueue are FIFO queues, which are similar to LinkedList and ArrayList, but are more than synchronous List have better concurrency performance. Priorityblockingqueue is a prioritized queue that requires the implementation of comparable or the use of Comparator.

The last implementation is synchronousqueue, in fact he is not a real queue, it does not maintain storage space for the elements in the queue. With a direct delivery mechanism, put and take are blocked until a put action occurs after a take or take takes after a put.

Producer-Consumer model

Below we use the producer consumer model to build a Windows-like Indexing Service.

Package Org.bupt.xiaoye.chapter5_8;import Java.io.file;import Java.util.concurrent.blockingqueue;public class  Crawler implements Runnable {private final blockingqueue<file> b;private final File root; @Overridepublic void Run () {System.out.println ("Crawler begins to run!"); if (root = null| |!root.exists ()) return;crawl (root); System.out.println ("Crawler is shutdown!");} public void crawl (File root) {try {if (Root.isfile ()) {System.out.println ("crawling" + root); B.put (root);} else {for (File f:root.listfiles ()) {crawl (f);}}} catch (Interruptedexception e) {System.out.println (e);}} Public Crawler (blockingqueue<file> B, File root) {this.b = B;this.root = root;}}

Package Org.bupt.xiaoye.chapter5_8;import Java.io.file;import Java.util.concurrent.blockingqueue;public class Indexer implements Runnable {private final blockingqueue<file> blockingqueue; @Overridepublic void Run () { System.out.println ("Indexer begins to run!"); Try{while (True) {indexer (blockingqueue.take ());}} catch (Interruptedexception e) {System.out.println (e);} System.out.println ("Indexer is shutdown!");} private void indexer (file file) {System.out.println ("Indexing" +file.getabsolutepath ());} Public Indexer (blockingqueue<file> blockingqueue) {this.blockingqueue = Blockingqueue;}}

Package Org.bupt.xiaoye.chapter5_8;import Java.io.file;import Java.util.concurrent.blockingqueue;import Java.util.concurrent.synchronousqueue;public class Starter {public static final int BOUND = 1;public static void Main (Stri Ng[] args) throws interruptedexception{blockingqueue<file> Blockingqueue = new synchronousqueue<file> (); File File = new file ("E:\\baiduyundownload"); Crawler C1 = new Crawler (blockingqueue,file); Indexer Indexer = new Indexer (blockingqueue); thread T1 = new Thread (c1); Thread t2 = new Thread (indexer); T1.start (); T2.start (); T1.join (); T2.join ();}}

Double-ended queues-work-tight access

Java6 adds two types of containers, Deque and Blockingdeque, which expand the Queue and Blockingqueue respectively. The Deque is a double-ended queue that enables efficient insertion and removal at the queue header and end of the queue. Specific implementations include Arraydeque and Linkedblockingdeque. The double-ended queue also applies to another correlation pattern, which is work -working stealing. In producer-consumer design, all consumers have a shared work queue, whereas in a work-tight design, each consumer has its own double-ended queue. If a consumer completes all the work in his own double-ended queue, it can secretly get work from the end of the other consumer's double-ended queue. The cryptographic work mode is more scalable than the traditional producer-consumer model because worker threads do not compete on a single shared task queue. Most of the time, they are simply accessing their own double-ended queues, which greatly reduces competition. When a worker thread needs to access another queue, it gets work from the tail of the queue instead of the head, thereby further reducing the level of contention on the queue.

Job keying is well suited to both consumer and producer issues-it can lead to more work when a job is performed. For example, when a web crawler processes a page, it usually finds that more pages need to be processed.

Blocking and interrupts

Threads can block or suspend execution for a number of reasons: waiting for the I/O operation to end, waiting for a lock to wake up from Thread.Sleep, or waiting for another thread to evaluate. When a thread is blocked, it is usually suspended and in a blocking state (BLOCKED, waiting, or timed_waiting). The difference between a blocking operation and a normal operation that takes a long time to execute is that the blocked thread must wait for a time that is not under its control to continue.

The put and take methods of Blockingqueue throw a check exception (Checked Exception) interruptedexception, which is the same as other methods in the class library, such as Thread.Sleep. When a method throws When interruptedexception, it means that the method is a blocking method. When a method throws Interruptedexception, there are two basic options:

passing interruptedexception to avoid this exception is usually the smartest strategy-just pass the interruptedexception to the caller of the method. The method of passing interruptedexception includes not capturing the exception at all or throwing the exception again after the capture. (typically if you call a blocking method in a custom method, you should re-throw it after you catch the interrupt exception for the blocking method)

recovery interrupts sometimes cannot throw interruptedexception, for example code is part of Runnable (because the Run method is defined to not throw any exceptions). In this case, you must capture interruptedexception and restore the interrupt state by calling the interrupt method of the current thread, so that the higher-level code in the call stack will see an interrupt thrown.

Java concurrency Programming (iv) blocking queues and producer-consumer patterns