Apache Nifi Component Development

Apache Nifi is an open source project by the United States National Security Agency (NSA) to the Apache Foundation, which is designed to automate the flow of data between systems. Thanks to its workflow-based programming philosophy, the Nifi is very easy to use, powerful, reliable and highly configurable. Two most important features are its powerful user interface and good data backtracking tools (official website blown)

Nifi is more convenient to use, Web-based control interface can be better management and control of data flow, data processing business process is also relatively clear, the following is how to customize the development of a component based on our own business

1. Engineering structure

Nifi to the engineering structure, we just create a project package in accordance with his format. Nar, put in. /lib/directory, startup services can be found in the Page Component center and use our controls

1) The simple way is to open a NAR module in the source code, modify the name according to our business and then ... (How to play how to play, save the trouble of Building engineering module)

2) is to create their own new, then generally three steps:

A. New Maven module Xxxx-xxx--bundle

Pom.xml describes dependencies:

B. Create a new Xxx-xxx-nar under the A module

Pom.xml Description Dependency Relationship

C. Create a new xxx-xxx-processors under the A module

Pom.xml Description Dependency Relationship

Concrete operation to see the structure of the source code to understand, this is not the point

2. Develop your own functional components

Components are generally divided into service and processor let's start with the following new processor

Generally in two steps 1. Component function Development, 2 component declaration (outward exposure)

component function development, in xxx--xxx-processors new engineering class Xxxx.java

Functional components are generally divided into three parts

A. external parameter declaration 2, output Relationship Declaration 3, Data flow trigger action

A. External parameter description

If our components need to interact with the outside world and need to be passed in externally, parameter declarations are required.

By declaring a parameter in this way, the Nifi itself provides a more validator that can be used to validate the parameters. We're watching.

After parameter definition needs to be exposed externally, the Getsupportedpropertydescriptors method needs to be overridden

Nifi also supports the dynamic addition of properties, in the same way that everyone is watching to play ...

2. Defining Conversion relationships

Nifi the flow of data through a transformation relationship, so we must define at least one transformation relationship in our component

The same relationship definition, need to be exposed to the outside, you need to rewrite getrelationships

Okay, until this preparation is basically done, go down to the main section and manipulate the file stream

3. File stream Operations

Nifi is flwofile as the processing unit, the data can be abstracted as a flowfile which is the data flow

Nifi data processing begins with a rewrite of Ontriger

The operation for Flowfile is broadly divided into three

1. Read the contents of the data read from the data stream

2. Write data to Flowfile

3. Finish reading and writing

After the logical processing is complete, Nifi must convert the Flowfile commit

All right, here we go. Component function Development, the following for our newly developed components to make a statement, so that its external exposure

Look, I'll just do it again.

All right, all done. Package and put it in the Nifi install directory under the Lib Directory restart service.

You see, it's so magical to beat.

All right, everybody, let's play, I'm just in touch with this, and it's not right or not in place much to forgive, together to communicate with each other to learn common progress, the first time to write a blog some more

All right, all right, end.

Apache Nifi Component Development