Storm series (1): build an environment for developing Storm topology in dotNet, dotnetstorm

Storm series (1): build an environment for developing Storm topology in dotNet, dotnetstorm

The previous blog compares the popular computing framework features. If you are a Java developer, you can choose one based on your business scenario. net developers, so none of them can be used out-of-the-box, at least before this article appeared. Based on the comparison in the previous article, we found that Storm should be a better framework for multi-language support, but even so, it does not provide a. Net adapter, nor can it find third-party open-source libraries on the Internet. So, Storm. Net. Adapter appears, an Adapter developed for Apache Storm using Csharp! Developed by myself, the project is open-source according to Apache License and Version 2.0.

This is the first article in the Storm series and will be updated later based on time!

Install Storm and the dependent environment to install Zookeeper

• Get the latest Zookeeper package: Official Website

• Decompress the package, copy zoo_sample.cfg under conf to zoo. cfg, and modify the configuration.

• Run bin \ zkServer. cmd in Windows and Run Command in Linux.bin/zkServer.sh start

Install Python, Java, and Maven

• Download Python 2.x for Installation

• To download and install JAVA 6 +, you must install JDK. Otherwise, an error occurs when you use Maven.

• Download and install Maven

Download Storm

• Get the latest Storm package: Official Website

• Decompress the package and modify the configuration in storm. yaml in conf.

Configure Environment Variables

• Configure Storm_Home and Java_Home; do not have spaces in the directory

• Add in classPath.;%JAVA_HOME%\lib\dt.jar;%JAVA_HOME%\lib\toos.jar;

• Add in path%STORM_HOME%\bin;%JAVA_HOME%\bin;

• Add the Maven directory to the path for ease of use.

Storm startup

• Start Zookeeper

• Runstorm nimbus(If Storm is not added to path, switch to the bin directory of Storm first, the same below)

• Runstorm supervisor(In the cluster environment, you can only execute this sentence if it is not the master)

• Runstorm ui, Monitoring Storm running status through http: // localhost: 8080/

Use Storm. Net. Adapter to obtain Storm. Net. Adapter

Currently, you can obtain the latest Storm. Net. Adapter library in the following ways:

• Compile your own version through source code: GitHub

• Download the compiled version and add it to reference: Release

• Use NuGet to obtain the latest version (recommended ):PM> Install-Package Storm.Net.Adapter

Create a sample project

• Reference Storm. Net. Adapter in the project to create Spout (based on ISpout) and Bolt (based on IBolt or IBasicBolt ).using Storm;

• Create a Java project managed by Maven and add the Topology corresponding to the dotNet Program

• On Windows (. Net Framework), you can call your Spout or Bolt in the following ways:

   super("cmd", "/k", "CALL", "StormSimple.exe", "generator");

• On Linux, Mac OSX, and Windows (mono) platforms, you can call your Spout or Bolt in the following ways:

   super("mono", "StormSimple.exe", "generator");

Packaging and Publishing

• Copy the compiled dotNet program to the resources directory and use the following Maven command to package your Topology:

   $ mvn package

• Use the Storm command line tool to submit the created Topology:

   $ storm jar storm-starter-*-jar-with-dependencies.jar storm.starter.WordCountTopologyCsharp wordcount

Storm Series

(1): build an environment for developing Storm topology in dotNet