flume log4j日誌接收__flume

flume 安裝與配置：

下載flume，然後解壓：

tar xvf apache-flume-1.5.2-bin.tar.gz -C ./

配置flume，在conf/flume-conf.properties下（沒有則建立，反正有template）:

# example.conf: A single-node Flume configuration# Name the components on this agenta1.sources = r1a1.sinks = k1a1.channels = c1# Describe/configure the sourcea1.sources.r1.type = avroa1.sources.r1.bind = localhosta1.sources.r1.port = 44446# Describe the sinka1.sinks.k1.type = logger# Use a channel which buffers events in memorya1.channels.c1.type = memorya1.channels.c1.capacity = 1000a1.channels.c1.transactionCapacity = 100# Bind the source and sink to the channela1.sources.r1.channels = c1a1.sinks.k1.channel = c1

啟動flume：

./bin/flume-ng agent --conf conf --conf-file conf/flume-conf.properties --name a1 -Dflume.root.logger=INFO,LOGFILE

flume java日誌執行個體

啟動flume後，可以建一個maven工程，pom.xml如下：

<?xml version="1.0" encoding="UTF-8"?><project xmlns="http://maven.apache.org/POM/4.0.0"         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">    <modelVersion>4.0.0</modelVersion>    <groupId>flume-test</groupId>    <artifactId>flume-test</artifactId>    <version>1.0</version>    <dependencies>        <dependency>            <groupId>org.slf4j</groupId>            <artifactId>slf4j-log4j12</artifactId>            <version>1.7.10</version>        </dependency>        <dependency>            <groupId>org.apache.flume</groupId>            <artifactId>flume-ng-core</artifactId>            <version>1.5.0</version>        </dependency>        <dependency>            <groupId>org.apache.flume.flume-ng-clients</groupId>            <artifactId>flume-ng-log4jappender</artifactId>            <version>1.5.0</version>        </dependency>    </dependencies></project>

注意依賴一定要有flume-ng-log4jappender，後面配置log4j要用。
建立一個執行個體：

package per.flume;import org.slf4j.Logger;import org.slf4j.LoggerFactory;import java.util.Date;/** * Created by hp on 15-3-16. */public class WriteLog {    protected static final Logger logger = LoggerFactory.getLogger(WriteLog.class);    public static void main(String[] args) throws Exception {        while (true) {            logger.info(String.valueOf(new Date().getTime()));            Thread.sleep(2000);        }    }}

resources下log4j配置如下：

### set log levels ###log4j.rootLogger=INFO, stdout, file, flumelog4j.logger.per.flume=INFO### flume ###log4j.appender.flume=org.apache.flume.clients.log4jappender.Log4jAppenderlog4j.appender.flume.layout=org.apache.log4j.PatternLayoutlog4j.appender.flume.Hostname=localhostlog4j.appender.flume.Port=44446### stdout ###log4j.appender.stdout=org.apache.log4j.ConsoleAppenderlog4j.appender.stdout.Threshold=INFOlog4j.appender.stdout.Target=System.outlog4j.appender.stdout.layout=org.apache.log4j.PatternLayoutlog4j.appender.stdout.layout.ConversionPattern=%d{yyyy-MM-dd HH:mm:ss} %c{1} [%p] %m%n### file ###log4j.appender.file=org.apache.log4j.DailyRollingFileAppenderlog4j.appender.file.Threshold=INFOlog4j.appender.file.File=./logs/tracker/tracker.loglog4j.appender.file.Append=truelog4j.appender.file.DatePattern='.'yyyy-MM-ddlog4j.appender.file.layout=org.apache.log4j.PatternLayoutlog4j.appender.file.layout.ConversionPattern=%d{yyyy-MM-dd HH:mm:ss} %c{1} [%p] %m%n

建立一個shell(run.sh)指令碼用於講flume的包加入classpath：

#!/bin/shjarlist=$(ls /home/hp/servers/apache-flume-1.5.2-bin/lib/*.jar)CLASSPATH=/home/hp/projects/flume-test/target/flume-test-1.0.jarfor jar in ${jarlist}do    CLASSPATH=${CLASSPATH}:${jar}doneecho ${CLASSPATH}java -classpath $CLASSPATH per.flume.WriteLog

build 項目：

mvn clean package

我這裡產生的是flume-test-1.0.jar,然後啟動shell:

./run.sh

此時進入apache-flume-1.5.2-bin/logs/flume.log,可以看到我們程式打的日誌了。