Introduction to windows--based on Flink

Source: Internet
Author: User
Tags apache flink

The new Year, the new beginning, the new habit, now begins.

  1. Introduction

  Flink, a German company named Dataartisans, was formally promoted to the top project by Apache in 2016 (with open source architectures like spark and Storm). And in the past 2016 years, a total of 10 versions, including 1.0.0, have been released, and the speed of development is conceivable. This study is the core of Flink feature,windows.

Dataartisans Official website:

http://data-artisans.com/

Apache Flink official Website:

http://flink.apache.org/

2.windows  2.1 What Flink windows

People tend to process data by converting streaming data into batches and then processing it, such as some cumulative statistics. In the face of a steady stream of flow, this approach derives a new noun-windows. is a typical Windows operation.

  

2.2 Why Flink Windows  

Flink provides a set of easy-to-use and flexible extensible Windows interface, from window type can be divided into sliding window, fixed window, Session window, from the window properties can be divided into the event window, time window.

2.3 How Flink windows

The window of Flink is divided into three parts, which are windowassigner,trigger,evictor in order of operation. Explanations provided for the official website

  

Data comes in first by Windowassigner to different windows, and one data can exist in multiple windows at the same time. Each window has a trigger to determine whether the data should be purge or evaluate. When the data is evaluate, it goes into the evictor (I feel similar to the filter). The user can customize the processing logic at evaluate, similar to sum (), Min (), Max (), ReduceFunction FoldFunction orWindowFunction。

  Here is the Code combat

Disclaimer: All operating environments are based on Flink's official Flink Quickstart Job 1.1.3.

  

Val env = streamexecutionenvironment.getexecutionenvironment    //  create streams for names and Ages by mapping the inputs to the corresponding objects    val text = Env.sockettextstream ("127.0.0.1", 12580) 
    = Text.map {str=>      = Str.split (",")      (arr (0), arr (1). ToInt)    }      . Keyby (0)      . Countwindow (3)      . SUM (1)    counts.print ()    Env.execute ("Scala WordCount from Sockettextstream Example")

The test data source is the socket, which defaults to \ n as the stream delimiter, and the data sent by the server is:

a,1

a,2

b,1

b,1

a,3

b,1

The final program results are as follows:

(a,6)

(b,3)

This case is based on the native event countwindows provided by Flink, which already includes the Windowassigner,trigger,evictor

  Note:

The specified package must be imported or the class will not be found

   Error: (+) could not find implicit value for evidence parameter of type Org.apache.flink.api.common.typein Fo. typeinformation[(String, Int)]
Val counts = Text.map {str=>

  Join : Import org.apache.flink.streaming.api.scala._

  

  Resources:

  Apache official website Introduction of QuickStart

Https://ci.apache.org/projects/flink/flink-docs-release-1.1/quickstart/scala_api_quickstart.html

Apache official website Introduction of Flink windows

Http://flink.apache.org/news/2015/12/04/Introducing-windows.html

  

Introduction to windows--based on Flink

Related Article

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: info-contact@alibabacloud.com and provide relevant evidence. A staff member will contact you within 5 working days.

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.