Flume Study---Kafka source__flume

Kafka Source

Kafka source is an Apache Kafka consumer who reads messages from a Kafka topic. If you have multiple Kafka source runs, you can configure them using the same consumer group, so each will read a unique set of partitions in topic.
The main properties of the Type:source type must be used Org.apache.flume.source.kafka.KafkaSource kafka.bootstrap.servers:List of brokers in the Kafka cluster used by the source kafka.consumer.group.id: Unique identification of the consumer group. Setting the same IDs in multiple sources or agents means that they are part of the same consumer group Kafka.topics: Kafka Consumers who use all the good things separated from those topics get messages Kafka.topics.regex: Using regular expressions is a collection of topic that defines the source subscription, which has a higher precedence than kafka.topics and overrides Kafka.topics (if present). BatchSize: The maximum number of messages written to the channel in a batch- Note:

Kafka Source covers two Kafka user parameters:
Auto.commit.enable is set to "false" by the source, and each batch is committed. The Kafka source guarantees a message retrieval policy at least once. Duplicates can exist when the source is started.
Kafka Source also provides Key.deserializer (Org.apache.kafka.common.serialization.StringSerializer) and Value.deserializer ( Default value for Org.apache.kafka.common.serialization.ByteArraySerializer). Modifying these parameters is not recommended.
# # #启用的属性
Examples of topic lists with a well split subscription

Tier1.sources.source1.type = Org.apache.flume.source.kafka.KafkaSource
tier1.sources.source1.channels = Channel1
tier1.sources.source1.batchSize = 5000
Tier1.sources.source1.batchDurationMillis =
Tier1.sources.source1.kafka.bootstrap.servers = localhost:9092
tier1.sources.source1.kafka.topics = test1, Test2
tier1.sources.source1.kafka.consumer.group.id = custom.g.id

subscription topic with regular expression matching

Tier1.sources.source1.type = Org.apache.flume.source.kafka.KafkaSource
tier1.sources.source1.channels = Channel1
tier1.sources.source1.kafka.bootstrap.servers = localhost:9092
Tier1.sources.source1.kafka.topics.regex = ^topic[0-9]$
# The default kafka.consumer.group.id=flume is used

not to be continued ...