Introduction to windows--based on Flink

The new Year, the new beginning, the new habit, now begins.

　 1. Introduction

　　Flink, a German company named Dataartisans, was formally promoted to the top project by Apache in 2016 (with open source architectures like spark and Storm). And in the past 2016 years, a total of 10 versions, including 1.0.0, have been released, and the speed of development is conceivable. This study is the core of Flink feature,windows.

Dataartisans Official website:

http://data-artisans.com/

Apache Flink official Website:

http://flink.apache.org/

2.windows　　2.1 What Flink windows

People tend to process data by converting streaming data into batches and then processing it, such as some cumulative statistics. In the face of a steady stream of flow, this approach derives a new noun-windows. is a typical Windows operation.

　　

2.2 Why Flink Windows　　

Flink provides a set of easy-to-use and flexible extensible Windows interface, from window type can be divided into sliding window, fixed window, Session window, from the window properties can be divided into the event window, time window.

2.3 How Flink windows

The window of Flink is divided into three parts, which are windowassigner,trigger,evictor in order of operation. Explanations provided for the official website

　　

Data comes in first by Windowassigner to different windows, and one data can exist in multiple windows at the same time. Each window has a trigger to determine whether the data should be purge or evaluate. When the data is evaluate, it goes into the evictor (I feel similar to the filter). The user can customize the processing logic at evaluate, similar to sum (), Min (), Max (), ReduceFunction FoldFunction orWindowFunction。

　　Here is the Code combat

Disclaimer: All operating environments are based on Flink's official Flink Quickstart Job 1.1.3.

　　

Val env = streamexecutionenvironment.getexecutionenvironment    //  create streams for names and Ages by mapping the inputs to the corresponding objects    val text = Env.sockettextstream ("127.0.0.1", 12580) 
    = Text.map {str=>      = Str.split (",")      (arr (0), arr (1). ToInt)    }      . Keyby (0)      . Countwindow (3)      . SUM (1)    counts.print ()    Env.execute ("Scala WordCount from Sockettextstream Example")

The test data source is the socket, which defaults to \ n as the stream delimiter, and the data sent by the server is:

a,1

a,2

b,1

b,1

a,3

b,1

The final program results are as follows:

(a,6)

(b,3)

This case is based on the native event countwindows provided by Flink, which already includes the Windowassigner,trigger,evictor

　　Note:

The specified package must be imported or the class will not be found

　　 Error: (+) could not find implicit value for evidence parameter of type Org.apache.flink.api.common.typein Fo. typeinformation[(String, Int)]
Val counts = Text.map {str=>

　　Join : Import org.apache.flink.streaming.api.scala._

　　

　　Resources:

　　Apache official website Introduction of QuickStart

Https://ci.apache.org/projects/flink/flink-docs-release-1.1/quickstart/scala_api_quickstart.html

Apache official website Introduction of Flink windows

Http://flink.apache.org/news/2015/12/04/Introducing-windows.html

　　

Introduction to windows--based on Flink