1. Heartbeat detection, store supervisor and worker heartbeat (including their status), so that Nimbus can monitor the status of the entire cluster, thus restarting some of the dead worker;
2. Submit a Task
3. Store all state information for the entire cluster for the UI display
4. Storm's storage structure on ZK
/-{storm-zk-root}--The root directory of storm on zookeeper (default =/storm) | |-/assignments--Topology's task assignment Information |
| | |-/{topology-id}-This directory holds assignments information for each topology, including: corresponding Nimbus | --The Code directory, the start time of all tasks, the mapping of each task to the machine, and the port. Action for | -(assignments) to get the values of all assignments; and (Assignment-info Storm-id) | --To get the Assignmentinfo information for a given Storm-id | --The contents stored in Assignmentinfo are: | --: Executor->node+port:executor->start-time-secs:node->host | --specifically defined in the COMMON.CLJ | --(Defrecord assignment[master-code-dir node->host executor->node+port Executor
->start-time-secs]) | |-/storms--This directory holds the ID of all running topology |
| |
| | |-/{topology-id}--This file holds some information about this topology, including topoLogy's name, topology start running | --the time and the state of the topology. Operation (active-storms), get active under current path | --topology data. The saved content reference class Stormbase; (Storm-base Storm-id) gets the given | --stormbase data under Storm-id, specifically defined in COMMON.CLJ |
--(Defrecord stormbase [storm-name launch-time-secs status Num-workers component->executors]) | |-/supervisors--This directory holds all the supervisor Heartbeat Information |
| |
| | |-/{supervisor-id}-This file saves supervisor heartbeat information including: Heartbeat time, hostname, this supervisor | --Worker's port number, run time (specifically see Supervisorinfo Class). Operation (supervisors) Get | --All supervisors nodes; (Supervisor-info Supervisor-id) get a given |
--Supervisor-id corresponding supervisorinfo information; specifically defined in COMMON.CLJ | |
--(Defrecord supervisorinfo [time-secs hostname assignment-id used-ports meta Scheduler-meta Uptime-secs])
| |-/wOrkerbeats--The heartbeat of all worker |
| | |-/{topology-id}--This directory holds all the worker's heartbeat information for this topology |
| |
|-/{supervisorid-port}--The heartbeat information of the worker, including the heartbeat time, worker run time, and some statistics | | --operation (heartbeat-storms) to get all topology with Heartbeat data | --(Get-worker-heartbeat storm-id node port) Get specific one topology under | --the heartbeat condition of a worker (Node:port), |
--(Executor-beats Storm-id executor->node+port) Get a executor heartbeat status |
|-/errors--All generated error information | |-/{topology-id}-This directory holds the topology error message below. Operation (error-topologies) Get error | -Topology, Errors Storm-id Component-id) get | --The error message under the given Storm-id Component-id |-/{component-id}