Overview:
- The DataStream program in Flink is a routine program that implements transformations on the data stream.
1. Demonstration procedures
Importorg.apache.flink.api.common.functions.FlatMapFunction;ImportOrg.apache.flink.api.java.tuple.Tuple2;ImportOrg.apache.flink.streaming.api.datastream.DataStream;Importorg.apache.flink.streaming.api.environment.StreamExecutionEnvironment;ImportOrg.apache.flink.streaming.api.windowing.time.Time;ImportOrg.apache.flink.util.Collector; Public classWindowwordcount { Public Static voidMain (string[] args)throwsException {streamexecutionenvironment env=streamexecutionenvironment.getexecutionenvironment (); DataStream<tuple2<string, integer>> DataStream =env. Sockettextstream ("LocalHost", 9999). FlatMap (NewSplitter ()). Keyby (0). TimeWindow (Time.seconds (5). SUM (1); Datastream.print (); Env.execute ("Window WordCount"); } Public Static classSplitterImplementsFlatmapfunction<string, Tuple2<string, integer>>{@Override Public voidFlatMap (String sentence, collector<tuple2<string, integer>> out)throwsException { for(String word:sentence.split ("") {out.collect (NewTuple2<string, integer> (Word, 1)); } } }}
NC-LK 9999
2. Data sources
- The program reads input from the source. The source can be attached to the program via Streamexecutionenvironment.addsource (sourcefunction).
- There are some pre-defined streaming data sources that are accessible in streamexecutionenvironment: , &NB Sp ReadTextFile (path) Step-by-line reading text files as strings & nbsp , &NB Sp , &NB Sp , REA Dfile (Fileinputformat, PATH) reads files through the specified file input format (the specified file, format) &NB Sp ReadFile (Fileinputformat, Path, WatChtype, Interval, pathfilter, typeinfo) This is a method that is called internally by the first two methods. It reads the file under path based on the given Fileinputformat, based on the provided
watchType,这个源会定期监测(每 interval ms)新数据的路径。
- Socket-based Sockettextstream are read from sockets. Elements can be separated by a delimiter.
- Collection-based &NBSP ; , &NB Sp , &NB Sp , &NB Sp Fromcollec tion (Collection) Create a data stream from Java Java.util.Collection, all elements in the collection must be of the same type. fromcollection (Iterator, Class) Create a data stream from an iterator, class specifies the data type of the element returned by the iterator. fromelements (T ...) Create a data stream from the sequence of a given object, all objects must be of the same type。 , &NB Sp fromparallelcollection (Splittableiterator, Class) In parallel execution, create a data stream from an iterator, class specifies the data type of the element returned by the iterator. , &NB Sp , &NB Sp , &NB Sp , &NB Sp generatesequence (from, to) &NBSP ; generates a sequence of numbers within a given time interval, executed in parallel.
- The custom addsource is appended with a new source function. For example, to read from Apache Kafka, you can use Addsource (new flinkkafkaconsumer08<> (...)).
3.DataStream transformations reference operator.
4.Data Sinks Data Reception
Apache flink-streaming (DataStream API)