Flume ng Source Code Analysis (iii) using the event interface to represent the data flow

Flume Ng has 4 main components:

Event represents the flow of data passed between flume agents

Source indicates that the event data stream is received from an external source and then passed to the channel

Channel represents the temporary storage of the event data stream passed from source

Sink represents the receipt of stored event traffic from the channel and is passed to the downstream source or endpoint warehouse

This article looks at the data flow represented by the event interface. The source, Channel, and sink data streams are encapsulated based on the event interface.

Public interface Event {  /**   * Returns A map of name-value pairs describing the data stored in the body.   *  /Public map<string, string> getheaders ();  /**   * Set the event headers   * @param headers Map of headers to replace the current headers.   *  /public void Setheaders (map<string, string> headers);  /**   * Returns The raw byte array of the data contained in this event.   *  /Public byte[] GetBody ();  /**   * Sets the raw byte array of the data contained in this event.   * @param body the data.   *  /public void Setbody (byte[] body);}

The event interface is very simple and the data flow is divided into two parts: the message header and the message body. The message header is a key-value that stores the structure of the string. The message body is a normal byte array.

The class hierarchy for event is as follows

Take a look at the specific implementations of commonly used simpleevent, which are used by components such as Execsource to encapsulate local log data. Its implementation is very simple, is to set the header and body two parts of the data.

public class SimpleEvent implements Event {  private map<string, string> headers;  Private byte[] body;  Public simpleevent () {    headers = new hashmap<string, string> ();    BODY = new byte[0];  }  @Override public  map<string, string> getheaders () {    return headers;  }  @Override public  void Setheaders (map<string, string> headers) {    this.headers = headers;  }  @Override public  byte[] GetBody () {    return body;  }  @Override public  void Setbody (byte[] body) {    if (BODY = = null) {      BODY = new byte[0];    }    This.body = body;  }  @Override public  String toString () {    Integer bodylen = null;    if (BODY = null) Bodylen = body.length;    Return "[Event headers =" + Headers + ", Body.length =" + Bodylen + "]";  }}

See how to create an Event object instance

public class EventBuilder {  /**   * Instantiate a Event instance based on the provided body and headers.   * If <code>headers</code> is <code>null</code> and then it's ignored.   * @param body   * @param headers   * @return   *  /public static Event Withbody (byte[] body, map<string, Stri ng> headers) {    Event event = new SimpleEvent ();    if (BODY = = null) {      BODY = new byte[0];    }    Event.setbody (body);    if (headers! = null) {      event.setheaders (new hashmap<string, string> (headers));    }    return event;  }}

Flume ng Source Code Analysis (iii) using the event interface to represent the data flow