Multiplexing technology is intended to send an event to a specific channel based on configuration information.
A source instance can specify multiple channels, but a sink instance can only specify one channel.
Flume supports fanning out the flow from one source to multiple channels. There is modes of fan out, replicating and multiplexing
Flume supports two modes of output from source to multiple channel: copy and Reuse
1. In copy mode, the event data received by source is output to all channel configured
2. In multiplexing mode, the event data received by source is output to a specific channel configured
The first choice of this mode, flume through the selector selector, the default is the copy mode
# list The sources, sinks and channels in the agentagent_foo.sources = Avro-appsrv-source1agent_foo.sinks = hdfs-cluster1- Sink1 avro-forward-sink2agent_foo.channels = mem-channel-1 file-channel-2# set channels for Sourceagent_ Foo.sources.avro-appsrv-source1.channels = mem-channel-1 file-channel-2# set channel for Sinksagent_ Foo.sinks.hdfs-cluster1-sink1.channel = Mem-channel-1agent_foo.sinks.avro-forward-sink2.channel = file-channel-2# Channel Selector configurationagent_foo.sources.avro-appsrv-source1.selector.type = Multiplexingagent_ Foo.sources.avro-appsrv-source1.selector.header = stateagent_foo.sources.avro-appsrv-source1.selector.mapping.ca = Mem-channel-1agent_foo.sources.avro-appsrv-source1.selector.mapping.az = File-channel-2agent_ Foo.sources.avro-appsrv-source1.selector.mapping.ny = Mem-channel-1 File-channel-2agent_ Foo.sources.avro-appsrv-source1.selector.default = Mem-channel-1
Think of this is a multiplexed channel mode, where the header is the header of the event data, according to the value of the header to determine the output to which channel.
Of course, the above scenario applies to the case where the data source collection code is coupled to its own application code, because you cannot determine the contents of the header data for this event until the data is collected by other means without the code involved in the decoupling.
Copy mode is much simpler, slightly!
Multiplexing technology for "Flume" Flume multiplexing