Flume reading data from Kafka to HDFs configuration

#the name of sourceAgent.sources =Kafkasource#the name of channels, which is suggested to be named according to typeAgent.channels =Memorychannel#Sink's name, suggested to be named according to the targetAgent.sinks =Hdfssink#Specifies the channel name used by SourceAgent.sources.kafkaSource.channels =Memorychannel#Specify the name of the channel that sink needs to use, Note that this is the channelAgent.sinks.hdfsSink.channel =Memorychannel#--------kafkasource related configuration-----------------#defining message Source TypesAgent.sources.kafkaSource.type =Org.apache.flume.source.kafka.KafkaSource#define the address of the ZK where Kafka is located##Special attention Here: is the address of the Kafka zookeeper#Agent.sources.kafkaSource.zookeeperConnect = 127.0.0.1:2181#Configure the consumption of Kafka topic#agent.sources.kafkaSource.topic = Testtopic#Configure the ID of the consumer groupAgent.sources.kafkaSource.groupId =Flume#consumption time-out, refer to the following notation to configure all other Kafka consumer Options. Note that the format starts from KAFKA.XXX and is consumer configuration propertyagent.sources.kafkaSource.kafka.consumer.timeout.ms = 100#-------memorychannel related configuration-------------------------#Channel TypeAgent.channels.memoryChannel.type =Memory#event capacity for channel storageagent.channels.memorychannel.capacity=10000#Transaction Capacityagent.channels.memorychannel.transactioncapacity=1000#---------hdfssink related configuration------------------Agent.sinks.hdfsSink.type =HDFs#Note that we output to one of the following subfolders in DataxAgent.sinks.hdfsSink.hdfs.path = hdfs://lenovo:9000/user/hive/warehouse/test/%y%m%d%HAgent.sinks.hdfsSink.hdfs.writeFormat =Textagent.sinks.hdfsSink.hdfs.fileType=DataStreamagent.sinks.hdfsSink.hdfs.rollSize= 1024Agent.sinks.hdfsSink.hdfs.rollCount= 0Agent.sinks.hdfsSink.hdfs.rollInterval= 60#configuring prefixes and suffixesAgent.sinks.hdfssink.hdfs.fileprefix=testAgent.sinks.hdfsSink.hdfs.fileSuffix=. Data#avoid using temporary files before closing filesagent.sinks.hdfssink.hdfs.inuserprefix=_agent.sinks.hdfssink.hdfs.inusersuffix=#Custom Interceptorsagent.sources.kafkasource.interceptors=I1agent.sources.kafkaSource.interceptors.i1.type=com.hadoop.flume.formatinterceptor$Builder

Flume reading data from Kafka to HDFs configuration