Flume (NG) custom sink implementation and attribute injection

Transferred from: http://www.aboutyun.com/thread-7884-1-1.html

Questions Guide:
1. How to implement the Flume end to customize a sink, to follow our rules to save the log.
2. To get the value of RootPath from the flume configuration file, how to configure it.






Recently you need to use Flume to do the collection of remote logs, so learn some flume the most basic usage. This is a record only.

The overall idea of the remote log collection is that the remote custom implementation log4j Appender send the message to the Flume side, and the flume-side custom implements a sink to save the log according to our rules.

Custom Sink Code:

public class Localfilelogsink extends Abstractsink implements configurable {
private static final Logger Logger = loggerfactory
. GetLogger (Localfilelogsink. Class);
private static final String Prop_key_rootpath = "RootPath";
Private String RootPath;
@Override
public void Configure (context context) {
String RootPath = context.getstring (Prop_key_rootpath);
Setrootpath (RootPath);
}

@Override
Public Status process () throws Eventdeliveryexception {
Logger. Debug ("do process");
｝
The copy code implements the configurable interface to get the values of the configured parameters from the context at initialization time by means of the Configure method. Here, we want to get the value of RootPath from the flume configuration file, which is the root path of the log save. The configuration in Flume-conf.properties is as follows:
Agent.sinks = Loggersink
Agent.sinks.loggerSink.rootPath =./logs Copy Code
Loggersink is the name of the custom sink, we take the value when the key, only need to loggersink the back part, that is, the rootpath here.

The execution of the actual business logic is implemented by inheriting the process method in the replication Abstractsink. The channel is obtained from the Getchannel method of the base class, and the event processing is removed from it.

Channel ch = getchannel ();
Transaction Txn = Ch.gettransaction ();
Txn.begin ();
try {
Logger. Debug ("Get event.");
Event event = Ch.take ();
Txn.commit ();
Status = Status. Ready;
return status;
}finally {
Log. Info ("Trx close.");
Txn.close ();
Copy the Code