Apache flink-streaming (DataStream API)

Overview:

• The DataStream program in Flink is a routine program that implements transformations on the data stream.

1. Demonstration procedures

Importorg.apache.flink.api.common.functions.FlatMapFunction;ImportOrg.apache.flink.api.java.tuple.Tuple2;ImportOrg.apache.flink.streaming.api.datastream.DataStream;Importorg.apache.flink.streaming.api.environment.StreamExecutionEnvironment;ImportOrg.apache.flink.streaming.api.windowing.time.Time;ImportOrg.apache.flink.util.Collector; Public classWindowwordcount { Public Static voidMain (string[] args)throwsException {streamexecutionenvironment env=streamexecutionenvironment.getexecutionenvironment (); DataStream<tuple2<string, integer>> DataStream =env. Sockettextstream ("LocalHost", 9999). FlatMap (NewSplitter ()). Keyby (0). TimeWindow (Time.seconds (5). SUM (1);        Datastream.print (); Env.execute ("Window WordCount"); }     Public Static classSplitterImplementsFlatmapfunction<string, Tuple2<string, integer>>{@Override Public voidFlatMap (String sentence, collector<tuple2<string, integer>> out)throwsException { for(String word:sentence.split ("") {out.collect (NewTuple2<string, integer> (Word, 1)); }        }    }}

NC-LK 9999

2. Data sources

• The program reads input from the source. The source can be attached to the program via Streamexecutionenvironment.addsource (sourcefunction).
• There are some pre-defined streaming data sources that are accessible in streamexecutionenvironment:                                               ,         &NB Sp                     ReadTextFile (path)     Step-by-line reading text files as strings   & nbsp                          ,         &NB Sp                          ,         &NB Sp                          ,         REA Dfile (Fileinputformat, PATH)     reads files through the specified file input format (the specified file, format)         &NB Sp                           ReadFile (Fileinputformat, Path, WatChtype, Interval, pathfilter, typeinfo)       This is a method that is called internally by the first two methods. It reads the file under path based on the given Fileinputformat, based on the providedwatchType，这个源会定期监测(每 interval ms)新数据的路径。
• Socket-based Sockettextstream are read from sockets. Elements can be separated by a delimiter.
• Collection-based                                 &NBSP ;                          ,         &NB Sp                          ,         &NB Sp                          ,         &NB Sp                                 Fromcollec tion (Collection)       Create a data stream from Java Java.util.Collection, all elements in the collection must be of the same type.             fromcollection (Iterator, Class)       Create a data stream from an iterator, class specifies the data type of the element returned by the iterator.                           fromelements (T ...)        Create a data stream from the sequence of a given object, all objects must be of the same type。                          ,         &NB Sp         fromparallelcollection (Splittableiterator, Class)     In parallel execution, create a data stream from an iterator, class specifies the data type of the element returned by the iterator.                          ,         &NB Sp                          ,         &NB Sp                          ,         &NB Sp                          ,         &NB Sp                       generatesequence (from, to)     &NBSP ;   generates a sequence of numbers within a given time interval, executed in parallel.
• The custom addsource is appended with a new source function. For example, to read from Apache Kafka, you can use Addsource (new flinkkafkaconsumer08<> (...)).

3.DataStream transformations reference operator.

4.Data Sinks Data Reception

Apache flink-streaming (DataStream API)