Twitter Storm: configure the development environment

This article describes how to configure a storm development environment. The following steps are taken:

1. Download the storm release version, decompress it, and add the bin/directory to the environment variable PATH.
2. In order to enable/disable topology on the Remote storm cluster, configure the cluster information in~/.storm/storm.yaml.

The following is a detailed description.

What is the storm development environment like?

Storm has two operation modes: Local Mode and remote mode. When using the local mode, you can develop and test your topology on your local machine, and everything is simulated on your local machine; in the remote mode, the topology you submit will be executed on the machine in the cluster.

In a storm development environment, you have installed topology for development and testing in local mode. You have packaged topology to deploy it to a remote cluster. Submit and terminate all things required by topology on the remote cluster.

Let's take a quick look at the relationship between your machine and the Remote storm cluster. Storm clusters are managed by a control node called Nimbus. Your machine communicates with nimbus to submit the topology code and run this topology, while nimbus automatically distributes your topology code within the cluster and assigns tasks to each machine. Your machine uses a client called storm to communicate with nimbus. Storm is useful only in the remote mode. It is useless for the development and test of topology in the local mode.

Install a storm on a Local Machine

If you want to submit topology from your machine to a Remote storm cluster, you should install a storm release on your local device. After the storm release is installed, you will get the tool for communicating with the remote cluster: storm. To install storm locally, download the code from here and decompress it to a directory on your machine. Then add the bin/directory to the environment variable PATH and grant the bin/storm executable permission.

Installing a storm release is only used to communicate with Remote storm clusters. To develop and test topology in local mode, we recommend that you use maven to use storm as a dev dependency for your project. For how to use Maven to develop the storm project, read this article: Maven.

Start/end topology in a remote Cluster

In the previous step, the storm client that can communicate with the remote cluster is installed on your machine. Now you only need to tell your storm client which cluster to communicate. Specify the address of the control node of your cluster in ~ /. Storm/storm. yaml:

1	nimbus.host: "123.45.678.890"

Alternatively, you can use the storm-deploy project to operate storm clusters on AWS. It automatically sets your~/.storm/storm.yamlFile. You can manually "add" a cluster (or switch between multiple clusters)-use the "attach" command:

1	lein run :deploy --attach --name mystormcluster