07_flume_regex Interceptor Practice

Practice One: Regex filter interceptor1, target scene

The function of the Regex filter Interceptor:

1) match the contents of the event body with the regular expression specified in the configuration
2) If the content matches, the event is discarded
3) If the content does not match, the event is released

2. Flume Agent configuration file

# onDefine agent name, source/sink/Channel A1.sources=r1a1.sinks=K1a1.channels=c1# GenevaSource,http,jsonhandlera1.sources.r1.type=Httpa1.sources.r1.bind=Mastera1.sources.r1.port=6666A1.sources.r1.handler=Org.apache.flume.source.http.JSONHandler# Interceptor regex filter, match event body for filtera1.sources.r1.interceptors = I1 A1.sources.r1.interceptors.i1.type = Regex_filter A1.sources.r1.interceptors.i1.regex = ^[0-9]*$ # filter Match Ed Event a1.sources.r1.interceptors.i1.excludeEvents = True  # GenevaLogger Sinka1.sinks.k1.type=logger# toChannel,memorya1.channels.c1.type=memorya1.channels.c1.capacity= +a1.channels.c1.transactionCapacity= -#  .bind Source,sink to Channela1.sources.r1.channels=C1a1.sinks.k1.channel= C1

3. Verify Regex Filter Interceptor

1) Send HTTP requests with different body via curl-x post-d ' JSON data, 1 of which meet the Regex

2) Watch the terminal print out the event,body 1234 of the event is filtered, and does not appear

4. Regex Filter Interceptor Official documentation

Practice two: Regex extractor interceptor1, target scene

The role of the Regex extractor Interceptor:
1) match the contents of the event body with the regular expression specified in the configuration
2) If the content matches, the Key:value is added to the header of the event with the given key in the configuration file.
3) The contents of the event body will not change

2, Flume agent configuration file

# onDefine agent name, source/sink/Channel A1.sources=r1a1.sinks=K1a1.channels=c1# GenevaSource,http,jsonhandlera1.sources.r1.type=Httpa1.sources.r1.bind=Mastera1.sources.r1.port=6666A1.sources.r1.handler=Org.apache.flume.source.http.JSONHandler# Geneva Regex Extractor Interceptor,match event body to extract character and digital  A1.sources.r1.interceptors =  i1  a1.sources.r1.interceptors.i1.type = regex_extractora1.sources.r1.interceptors.i1.regex = (^[a-za-z]*) \\s ([0-9]* $) # Regex matches and groups, matching results will have two parts, note \s whitespace characters to escape # Specify key For 2 matched parta1.sources.r1.interceptors.i1.serializers = S1 s2# key Namea1.sources.r1.interceptors.i1.serializers.s1.name = Worda1.sources.r1.interceptors.i1.serializers.s2.name =   Digital #GenevaLogger Sinka1.sinks.k1.type=logger# toChannel,memorya1.channels.c1.type=memorya1.channels.c1.capacity= +a1.channels.c1.transactionCapacity= -#  .bind Source,sink to Channela1.sources.r1.channels=C1a1.sinks.k1.channel= C1

3. Verify Regex Extractor Interceptor

1) HTTP request is sent via Curl-x post-d ' JSON data ', thecontents of body is "Shayzhang 1234", where shayzhang,1234 will be matched by regular expression

2) Watch logger Print to the event,header of the terminal will add two parts Word:shayzhang, digital:1234

07_flume_regex Interceptor Practice