The Acker mechanism of storm is understood.

Reprint please specify original address http://www.cnblogs.com/dongxiao-yang/p/6142356.html

The Storm topology has some special tasks called "Acker" that are responsible for tracking the DAG of each tuple emitted by each Spout. There are three prerequisites to open the Storm tracker mechanism:

1. When spout emit a tuple, add the 3rd parameter MessageID
2. The number of Acker in the configuration is at least 1
3. When Bolt emit, add the second parameter anchor tuple to keep the tracker link.

When a tuple is created in the topology, either in Spout or in Bolt, the tuple is configured with a random 64-bit ID. Acker is a tuple DAG that uses these IDs to track each spout tuple. Here is an example of an ACK mechanism in storm source analysis.

Understand how the whole rhythm is divided into parts.

Steps 1 and 2 spout send a message to BOLT1 and BOLT2, step 3 means spout emit to Acker Bolt to register this root message, the ACK value is set to the message sent by the corresponding 64-bit ID of the XOR operation value, corresponding to the t1^ T2.

Step 4 indicates that BOLT1 received T1, a single tuple was split into three messages T3t4t5 sent to BOLT3. Step 6 BOLT1 submits the ACK value of t1^t3^t4^t5 to the Acker Bolt when the ACK () method is called.

None of the bolts in steps 5 and 7 produce new messages, so ACK () commits the ACK values of T2 and t3^t4^t5 to Acker Bolt respectively.

In summary, the spout generated by the tuple tree corresponding to the ACK value of the operation is T1^T2^T1^T3^T4^T5^T2^T3^T4^T5 according to the rules of the XOR operation, the ACK value is exactly zero.

Step 8 for Acker Bolt found that the root spout final corresponding ACK is 0 after all the derived data has been processed successfully, it will notify the corresponding spout,spout will invoke the corresponding Ack method.

Storm's implementation of this mechanism ensures that no matter how large a tuple is, a message corresponding to the ACK value occupies a fixed amount of space, greatly saving memory space.

Reference documents

1 Storm ack and fail mechanism re-discussion

2 Storm Application Series--reliability and Acker mechanism

3 "Strom Source Analysis" 12th Chapter

The Acker mechanism of storm is understood.