Storm-a little thought in the process of using

Introduction

These days in order to optimize the original data processing framework, compare the system to learn some of the storm's content, sorting out the experience

1. Storm provides a data-processing idea that does not provide a specific solution

The core of storm is the definition of topo, and Topo carries all the business logic, and we orchestrate the private business implementation logic based on the storm's topo organizational plan.

The business abstraction of storm is the most basic paradigm of information flow processing.

2. Storm supports multiple languages

Storm provides a processing framework that supports multiple languages, and Java is just one, and we can implement it in different languages topo

3. Storm will enter the same class as the dependent class

This may be strange for beginners, but this is the compromise that storm has made for distributed deployments, and if the jar package itself has everything that is needed to run the runtime, then storm itself can exist as a running framework, and the distribution of jar packages between cluster can be much easier.

4. Storm can support multiple topo

Over and over again, Storm itself is just a processing framework for data, and the core of business logic is topo, where we can submit multiple topo to the storm cluster to take full advantage of Storm's clustered processing power. Also, because the configuration is valid for Topo, you can also define the processing power of the topo.

5. Storm is better at instant processing compared to Hadoop

Storm does not have a complex and lengthy preprocessing process for Hadoop, and distributed storm and distributed Hadoop are a good pair of partners who can collectively form a large data processing framework.

6. Storm's topo communication is basically not feasible

It is not feasible for Topo to run independently in storm, without interference, and topo direct communication at the code level (at least I have not found), and can only be monitored through the Storm UI and limited APIs.

If there is a communication demand, you can consider using zookeeper or database as the intermediary of communication, there is topo and intermediary communication to achieve indirect communication.

Storm-a little thought in the process of using