Java Concurrency Programming framework Disruptor

What is Disruptor?

Disruptor is a high-performance asynchronous processing framework, a lightweight JMS, similar to the blockingqueue in the JDK, but it is very fast to handle, winning the 2011 Program Framework Innovation Award, claiming that " one thread can handle 600W orders in a second." "(This is scary), and disruptor is not just buffer, it provides a very powerful function, such as it can help us to easily build data flow processing (such as a data first to A and B these 2 consumers in parallel processing in the C processing, is not a bit think of storm this stream processing, actually strom the bottom of the application of disruptor to achieve the communication between workers. This article will be described using the latest version of Disruptor 3.3.6, which can be used in https://github.com/LMAX-Exchange/disruptor/releases Download the latest jar pack and start the Disruptor tour.

Tyre: Ringbuffer

Ringbuffer, a ring buffer, plays a very important role in disruptor, and understanding the structure of Ringbuffer helps us understand why disruptor is so fast, unlocked, and the implementation details of the producer/consumer model . As shown in the following:

650) this.width=650; "src=" Http://s1.51cto.com/wyfs02/M01/8B/F8/wKioL1heKq_wpmu8AAA9zSBGoCQ743.png "title=" Sogou _ 2016-12-24_15-59-04.png "alt=" Wkiol1hekq_wpmu8aaa9zsbgocq743.png "/>

Array

This tire-like thing is actually an array, the advantage of using arrays is of course because of the preload so that access is much faster than the linked list.

Serial number

How do you understand that the element in Ringbuffer has the concept of ordinal and the serial number is always growing? For example, the Ringbuffer size is 10, then the sequence number starts from 0, and when it is 9, it is equivalent to a circle, and if it continues to grow, it will cover the No. 0 element. That is to say by ordinal%size to locate the element, realize set/get operation. Here also found a different way with the queue, that is, there is no deletion of the element, only cover, in fact, this special ring storage ringbuffer, so that it does not take a lot of time for memory cleanup/garbage collection.

Because it involves the modulo operation, for the CPU to make bit operations more efficient, the size of the Ringbuffer should be 2 of the n-th side.

Lock-free mechanism

In producer/consumer mode, Disruptor is known as the " lock-free parallel Framework " (to know that Blockingqueue is implemented using the lock lock mechanism), how is this done? Let's take a detailed analysis of the following:

A producer + a consumer

The producer maintains a production pointer p, the consumer maintains a consumer pointer C, of course p and C are essentially serial numbers. 2 Each operation of each, do not need to lock, just need to pay attention to the speed of producers and consumers, of course, this in the disruptor has done for us, is to judge the P and C can not exceed the size of a lap.

A producer + multiple consumers

Multiple consumers of course hold multiple consumer pointers c1,c2, ..., consumers according to C for their respective reading data, only need to ensure that the speed of producers "co-ordination" of the slowest consumer speed, is that can not go beyond the concept of a circle. There is no need to lock at this time.

Multiple producers + N Consumers

Obviously, no matter how many producers have, the producer pointer p can only exist one, otherwise the data will be messed up. So many producers share a P-pointer, in the disruptor is actually the use of CAS mechanism to ensure that multi-threaded data security, and no use of locks.

Disruptor First experience: simple producers and consumers

Business Data Object Pojo (Event)

public class order {    //Order Id    private long  id;    //Order Information     private String info;     //Order Price     private double price;    public  Long getid ()  {        return id;     }    public void setid (Long id)  {         this.id = id;    }    public string  getinfo ()  {        return info;     }    public void setinfo (String info)  {         this.info = info;    }    public  double getprice ()  {&NBsp;       return price;    }     public void setprice (Double price)  {         this.price = price;    }}

Business Data Factory (Factory)

public class Orderfactory implements eventfactory{@Override public Object newinstance () {System.out.println        ("Orderfactory.newinstance");    return new Order (); }}

Event handlers (Handler, consumer processing logic)

public class Orderhandler implements eventhandler<order>{@Override public void onEvent (order order, long L, b        Oolean b) throws Exception {System.out.println (Thread.CurrentThread (). GetName () + "Consumer Processing:" + L);        Order.setinfo ("info" + Order.getid ());    Order.setprice (Math.random ()); }}

Main

Public class main {    public static void main (String[]  args)  throws interruptedexception {        // Create Order Factory         OrderFactory orderFactory = new  Orderfactory (); Size of         //ringbuffer          int ringbuffer_size = 1024;        // Create disruptor        disruptor<order> disruptor =  New disruptor<order> (Orderfactory,ringbuffer_size,executors.defaultthreadfactory ());         //set event handlers   i.e. consumer          Disruptor.handleeventswith (New orderhandler ());         Disruptor.start ();     &nbsP;   ringbuffer<order> ringbuffer = disruptor.getringbuffer ();         //-------------production data         for (int  i = 0 ; i < 3 ; i++) {             long sequence = ringbuffer.next ();             order order = ringbuffer.get (Sequence);             order.setid (i);             ringbuffer.publish (Sequence);             system.out.println (Thread.CurrentThread (). GetName ()  +  "   Producer publishes a piece of data: " + sequence + "   Order ID: " + i";         } &nbSp;      thread.sleep (;        ) Disruptor.shutdown ();     }}

Operation Result:

650) this.width=650; "src=" Http://s2.51cto.com/wyfs02/M00/8B/FE/wKiom1heUkvDVrbbAABB1RkRt-k131.png "title=" Sogou _ 2016-12-24_18-48-24.png "alt=" Wkiom1heukvdvrbbaabb1rkrt-k131.png "/>

Description

In fact, the above results have been clearly explained, in the initial stage of the construction of Disruptor, will call the factory factory to instantiate the event data object in Ringbuffer.

In addition, when constructing disruptor, the API is used before 3.3.6:

650) this.width=650; "src=" Http://s4.51cto.com/wyfs02/M02/8B/FB/wKioL1heU5KD4yc1AABMgNq5GkY611.png "title=" Sogou _ 2016-12-24_18-53-43.png "alt=" Wkiol1heu5kd4yc1aabmgnq5gky611.png "/>

To 3.3.6 these APIs are deprecated, that is, it is no longer recommended to pass in a thread pool such as executor, but it is recommended to pass in the threadfactory thread factory. In this case, shutting down the disruptor automatically shuts down the executor thread pool without having to close the disruptor as it did before.

When constructing disruptor, it is important to note that Producertype (single or MULTI indicates whether it is an individual producer or multiple producers), Waitstrategy (policy selection, determines how consumers wait for the producer).

650) this.width=650; "src=" Http://s5.51cto.com/wyfs02/M00/8B/FB/wKioL1heVOLj98krAABdMOQG3ug461.png "title=" Sogou _ 2016-12-24_18-59-31.png "alt=" Wkiol1hevolj98kraabdmoqg3ug461.png "/>

Use Ringbuffer:workerpool alone

If the scene is relatively simple, we can simply use the Ringbuffer function instead of creating disruptor.

Public static void main (String[] args)  throws InterruptedException {     executorservice executor = executors.newfixedthreadpool (3);     ringbuffer<order> ringbuffer = ringbuffer.create (ProducerType.SINGLE,new  Orderfactory (), 1024,new yieldingwaitstrategy ());    workerpool<order>  Workerpool = new workerpool<order> (Ringbuffer,ringbuffer.newbarrier (),new  Ignoreexceptionhandler (), New orderhandler ());     workerpool.start (executor);     //-------------Production Data     for (int i = 0 ; i  < 30 ; i++) {        long sequence =  Ringbuffer.next ();         order order = ringbuffer.get ( sequence);     &Nbsp;   order.setid (i);         ringbuffer.publish ( sequence);         system.out.println (Thread.CurrentThread (). GetName ()  +  "  producer publishes a data:"  + sequence +  "  Order ID:"  + i);     }    thread.sleep (;        ) Workerpool.halt ();     executor.shutdown ();}

is actually using Workerpool to connect the consumer.

A producer + multiple consumers

650) this.width=650; "src=" Http://s5.51cto.com/wyfs02/M01/8B/FC/wKioL1heZFugNx-wAABXx8iDTG8613.png "title=" Sogou _ 2016-12-24_20-04-31.png "alt=" Wkiol1hezfugnx-waabxx8idtg8613.png "/>

Public static void main (String[] args)  throws InterruptedException {     //Create order Factory     OrderFactory orderFactory = new  Orderfactory (); Size of     //ringbuffer     int ringbuffer_size =  1024;    //Creating Disruptor    disruptor<order> disruptor  = new Disruptor<Order> (Orderfactory,ringbuffer_size,executors.defaultthreadfactory ());     //set event handlers   i.e. consumer     EventHandlerGroup<Order>  Eventhandlergroup = disruptor.handleeventswith (New orderhandler (), New OrderHandler2 ());     eventhandlergroup.then (New orderhandler3 ());     Disruptor.start ();    ringbuffer<order> ringbuffer =  Disruptor.getringbuffer ();     //-------------production data     for (int i = 0 ; i < 3 ; i++) {         long sequence = ringbuffer.next ();         order order = ringbuffer.get (Sequence);         order.setid (i);         Ringbuffer.publish (sequence);         system.out.println ( Thread.CurrentThread (). GetName ()  +  "  producer publishes a data:"  + sequence +  "  Order ID:"  + i);     }    thread.sleep (+);     Disruptor.shutdown ();}

Operation Result:

650) this.width=650; "src=" Http://s5.51cto.com/wyfs02/M01/8C/00/wKiom1heZJniEbQbAAA1jfoPs8I081.png "title=" Sogou _ 2016-12-24_20-05-29.png "alt=" Wkiom1hezjniebqbaaa1jfops8i081.png "/>

The producer produced 3 messages, one consumer thread 1 consumed the 3 data, the other consumer thread 2 also consumed the 3 data, 2 were parallel, when the consumer thread 1 and 2 were finished, 3 data were handed over to the consumer thread 3 for processing.

What if we want to follow a->b->c in order?

Disruptor.handleeventswith (New Handler1 ()).        Handleeventswith (New Handler2 ()). Handleeventswith (New Handler3 ());

What if we want a hexagonal operation?

650) this.width=650; "src=" Http://s4.51cto.com/wyfs02/M01/8B/FC/wKioL1heZe3QoLB9AACQcTxdeJ4198.png "title=" Sogou _ 2016-12-24_20-11-09.png "alt=" Wkiol1heze3qolb9aacqctxdej4198.png "/>

Handler1 H1 = new Handler1 ();        Handler2 h2 = new Handler2 ();        Handler3 h3 = new Handler3 ();        Handler4 h4 = new Handler4 ();        Handler5 h5 = new Handler5 ();        Disruptor.handleeventswith (H1, H2);        Disruptor.after (H1). Handleeventswith (H4);        Disruptor.after (H2). Handleeventswith (h5); Disruptor.after (H4, H5). Handleeventswith (H3);

To believe that you have some knowledge of disruptor, so many producers of multiple consumers how to achieve, in fact, and the above code very similar, nothing more than the producers are holding Ringbuffer can publish just.

This article is from the "Boundless Mind Infinite" blog, please be sure to keep this source http://zhangfengzhe.blog.51cto.com/8855103/1885830

Java Concurrency Programming framework Disruptor