Flume (3) Introduction to Netcatsource use of source components

First, overview:

This section first provides a data transfer process based on the Netcat Source+channel (memory) +sink (logger). Then dissect the code execution logic in Netcatsource.

Second, flume configuration file:

The following configuration file, netcat.conf, defines the source using Netcat, which listens on port 44444.

# Name The components in this agenta1.sources=r1a1.sinks=K1a1.channels=c1# Describe/Configure the Sourcea1.sources.r1.type=Netcata1.sources.r1.bind=Locahosta1.sources.r1.port=44444# Describe The Sinka1.sinks.k1.type=logger# use a channelwhichBuffers EventsinchMemorya1.channels.c1.type=memorya1.channels.c1.capacity= +a1.channels.c1.transactionCapacity= -# Bind The source and sink to the Channela1.sources.r1.channels=C1a1.sinks.k1.channel= C1

Three, command line start:

To switch to Flume's installation directory, execute the following code:

Bin/flume-ng agent--conf conf--conf-file study/netcat.conf--name a1-dflume.root.logger=info,console

Iv. using Telnet to directly access and send data:

Type the following code on the command line: where NODE5 is the host name where Flume resides.

44444

Enter information on the telnet command line:

The data received will be output at the start-up interface of the Flume:

As a result, the function of using netcat as source is demonstrated successfully.

In addition to using Telnet to send data, you can also implement a socket program to send data to the NODE5 host's 44444 port.

Of course, we found a problem, obviously the data sent in Telnet is: The is Flume Netcat source!, the received data is the is Flume ne. The data is incomplete. After analyzing the source code, see if you can find the reason.

The above display is incomplete because we are using the Loggersink component, and its internal implementation logic causes only 16 characters to be printed.

Five, agent start the basic steps:

Six, Netcatsource source analysis:

The full path of the class is Org.apache.flume.source.NetcatSource, inherits the Abstractsource and implements the configurable interface.

Because Netcatsource is a listener service, it is started by Eventdrivensourcerunner to invoke a thread, calling its start () method.

First, before formally starting source, the Configure method is executed first, and the parameters provided in the configuration file are initialized: Bind\port\ack-every-event\max-line-length.

The start () method is as follows:

In this method, a Accepthandler inner class instance is created, and the actual listener is implemented in the class's Run method.

Flume (3) Introduction to Netcatsource use of source components