Apache Flume Collector Installation

2, Flume Collector installation (through extends Abstractsink implements configurable, write directly to the database)2.1 Installation Environment

System:

CentOS Release 6.6

Software:

Flume-collector.tar.gz

2.2 Installation Steps2.2.1 Deploying flume Collector

Specific scripts (Jyapp users):

Cd/home/jyapp TAR-ZXVF flume-collector.tar.gz CD Flume-collector #修改flume环境变量 VI conf/flume-env.sh Specific configuration analysis: (Note: Red needs to be modified) ------------------------------------------------------ Export java_opts= "-xms1000m-xmx1000m-dflume.root.logger=info,logfile-dflume.monitoring.type=http- dflume.monitoring.port=34545-dcom.sun.management.jmxremote-djava.rmi.server.hostname= native IP - Dcom.sun.management.jmxremote.port=9999-dcom.sun.management.jmxremote.authenticate=false- Dcom.sun.management.jmxremote.ssl=false " ------------------------------------------------------------------------------------- #修改Collector脚本 VI conf/collector.conf Specific configuration analysis: (Note: Red needs to be modified) ------------------------------------------------------ Collector.sources = R1 Collector.channels = C1 C2 Collector.sinks = K1 K2 K3 K4 K5 K6 Collector.sources.r1.type = Avro Collector.sources.r1.bind = collect-side IP #本机IP Collector.sources.r1.port = Collector-bound port #本机接口消息端口 Collector.channels.c1.type = Memory Collector.channels.c1.capacity = 10000 Collector.channels.c1.transactionCapacity = 10000 Collector.channels.c1.byteCapacityBufferPercentage = 30 Collector.channels.c1.byteCapacity = 1048576000 Collector.channels.c2.type = Memory Collector.channels.c2.capacity = 10000 Collector.channels.c2.transactionCapacity = 10000 Collector.channels.c2.byteCapacityBufferPercentage = 30 Collector.channels.c2.byteCapacity = 1048576000 Collector.sinks.k1.type = Com.jy.modules.platform.flume.sink.OracleJdbcSink Collector.sinks.k1.url = Jdbc:oracle:thin: @IP:P ORT: Database name Collector.sinks.k1.user = User name Collector.sinks.k1.password = Password Collector.sinks.k2.type = Com.jy.modules.platform.flume.sink.OracleJdbcSink Collector.sinks.k2.url = Jdbc:oracle:thin: @IP:P ORT: Database name Collector.sinks.k2.user = User name Collector.sinks.k2.password = Password Collector.sinks.k3.type = Com.jy.modules.platform.flume.sink.OracleJdbcSink Collector.sinks.k3.url = Jdbc:oracle:thin: @IP:P ORT: Database name Collector.sinks.k3.user = User name Collector.sinks.k3.password = Password Collector.sinks.k4.type = Com.jy.modules.platform.flume.sink.OracleJdbcSink Collector.sinks.k4.url = Jdbc:oracle:thin: @IP:P ORT: Database name Collector.sinks.k4.user = User name Collector.sinks.k4.password = Password Collector.sinks.k5.type = Com.jy.modules.platform.flume.sink.OracleJdbcSink Collector.sinks.k5.url = Jdbc:oracle:thin: @IP:P ORT: Database name Collector.sinks.k5.user = User name Collector.sinks.k5.password = Password Collector.sinks.k6.type = Com.jy.modules.platform.flume.sink.OracleJdbcSink Collector.sinks.k6.url = Jdbc:oracle:thin: @IP:P ORT: Database name Collector.sinks.k6.user = User name Collector.sinks.k6.password = Password Collector.sources.r1.channels = C1 C2 Collector.sinks.k1.channel = C1 Collector.sinks.k2.channel = C1 Collector.sinks.k3.channel = C1 Collector.sinks.k4.channel = C2 Collector.sinks.k5.channel = C2 Collector.sinks.k6.channel = C2 Collector.sources.r1.selector.type = multiplexing Collector.sources.r1.selector.header = Flume.client.system.flag collector.sources.r1.selector.mapping.S000 = C1 collector.sources.r1.selector.mapping.S001 = C2 collector.sources.r1.selector.mapping.S002 = C1 collector.sources.r1.selector.mapping.S003 = C2 collector.sources.r1.selector.mapping.S004 = C1 collector.sources.r1.selector.mapping.S005 = C2 collector.sources.r1.selector.mapping.S006 = C1 collector.sources.r1.selector.mapping.S007 = C2 collector.sources.r1.selector.mapping.S008 = C1 collector.sources.r1.selector.mapping.S009 = C2 Collector.sources.r1.selector.default = C1 collector.sinkgroups = G1 G2 Collector.sinkgroups.g1.sinks = K1 K2 K3 Collector.sinkgroups.g1.processor.type = Load_balance Collector.sinkgroups.g1.processor.backoff = True Collector.sinkgroups.g1.processor.selector = Round_robin Collector.sinkgroups.g2.sinks = K4 K5 K6 Collector.sinkgroups.g2.processor.type = Load_balance Collector.sinkgroups.g2.processor.backoff = True Collector.sinkgroups.g2.processor.selector = Round_robin ------------------------------------------------------------------------------------- Background boot collector Cd/home/jyapp/flume-collector Bin/flume-ng agent--conf conf--conf-file conf/collector.conf--name Collector & Stop agent, use the KILL command

Apache Flume Collector Installation