Storm Series (i): Build dotnet environment for developing storm topologies

The previous blog compared the current popular computing framework features, if you are a Java developer, then choose according to the business scenario, but if you are a. Net developer, then none of the three can be used, at least before this article appears. Based on the comparison of the previous article, Storm should be a better framework for multi-lingual support, but even so, the official does not provide. Net adapters, and no third-party open source libraries are found on the web. So,storm.net.adapter appeared, an adapter for Apache Storm developed using Csharp! Project developed by me, according to Apache License, Version 2.0 open source.

This article is the "Storm series" of the first article, the latter will continue to update according to the time situation!

Install storm and dependent environment installation zookeeper

• Get the latest Zookeeper package: official website

• Unzip the package, copy conf under zoo_sample.cfg for ZOO.CFG, modify the relevant configuration

• Execute under bin\zkserver.cmd;linux directly under Windows environmentbin/zkServer.sh start

Install Python, Java and maven

• Download Python 2.x installation

• Download the JAVA 6+ installation, you must install the JDK version, or you will get an error using Maven

• Download Maven and install

Download storm

• Get the latest Storm package: official website

• After decompression, modify the relevant configuration in the Storm.yaml under Conf

Configuring Environment variables

• Configuration Storm_home and Java_home; The table of contents should not have space

• Increase in ClassPath.;%JAVA_HOME%\lib\dt.jar;%JAVA_HOME%\lib\toos.jar;

• Increase in Path%STORM_HOME%\bin;%JAVA_HOME%\bin;

• Add Maven directories to path for ease of use

Storm start

• Start Zookeeper

• Run storm nimbus (if you do not add storm to path, you need to switch to Storm's Bin directory, same as below)

• Run storm supervisor (in a clustered environment, the non-owner can only execute the sentence)

• Run storm ui , monitor Storm health through http://localhost:8080/

Get Storm.Net.Adapter using Storm.Net.Adapter

There are currently several ways to get the latest Storm.Net.Adapter library

• Compiling your own version with source code: GitHub

• Download the compiled version Add Reference: Release

• Get the latest version with NuGet (recommended):PM> Install-Package Storm.Net.Adapter

To create a sample project

• Referencing Storm.Net.Adapter in your project, creating Spout (Ispout-based) and bolts (based on Ibolt or Ibasicbolt) all requireusing Storm;

• Create a Java project that uses Maven management to increase the topology of the DotNet program

• Under the Windows (. Net Framework) platform, you can call your Spout or bolts in the following ways:

   super("cmd", "/k", "CALL", "StormSimple.exe", "generator");

• Linux, Mac OSX, Windows (mono) platform, you can call your Spout or Bolt in the following way:

   super("mono", "StormSimple.exe", "generator");

Packaging and Publishing

• Copy the compiled DotNet program to the resources directory and use the following Maven command to package your topology:

   $ mvn package

• Submit your created topology with the Storm command line tool:

   $ storm jar storm-starter-*-jar-with-dependencies.jar storm.starter.WordCountTopologyCsharp wordcount

Storm series Articles

(a): Build the environment for dotnet to develop storm topologies

Storm Series (i): Build dotnet environment for developing storm topologies