Storm [configuration items]-detailed explanation

Configuration items

Storm. zookeeper. servers zookeeper Server LIST

Storm. zookeeper. Port zookeeper connection Port

The local file system directory used by storm. Local. dir storm (which must exist and can be read and written by the storm process)

Storm. Cluster. Mode storm cluster running mode ([distributed | Local])

Whether to use zeromq as the message system in storm. Local. mode. zmq local mode. If it is set to false, Java Message System is used. The default value is false.

Storm. zookeeper. Root zookeeper

Storm. zookeeper. session. Timeout timeout when the client connects to zookeeper

Storm. ID indicates the ID of the topology in the running state. It consists of storm name and a unique random number.

Nimbus. Host Nimbus server address

Nimbus. Thrift. Port the thrift listening port of nimbus

Nimbus. childopts specifies the JVM option for the nimbus process during the storm-deploy project deployment

Nimbus. task. Timeout. secs heartbeat timeout time. After the timeout time, nimbus considers the task dead and assigns it to another address.

Nimbus. Monitor. freq. secs Nimbus checks the interval between heartbeat and reallocation tasks. Note that if the machine goes down, nimbus takes over and processes the task immediately.

Nimbus. Supervisor. Timeout. secs supervisor's heartbeat timeout time. Once Nimbus is exceeded, it considers the supervisor dead and stops distributing new tasks for it.

A special timeout setting for Nimbus. task. Launch. secs task startup. This value will be used before the first heartbeat after startup to temporarily replace nimbus. task. Timeout. secs.

Nimbus. Reassign whether Nimbus re-allocates and executes tasks when a task fails. It is true by default and is not recommended to be modified.

Nimbus. file. Copy. expiration. secs Nimbus determines the upload/download link timeout. When the idle time exceeds this setting, nimbus considers the Link Dead and actively disconnects

Ui. Port storm UI service port

DRPC. servers DRPC Server list to let drpcspout know who to communicate

DRPC. Port storm DRPC service port

Supervisor. slots. list of ports that can run workers on ports supervisor. each worker occupies one port, and each port runs only one worker. this configuration allows you to adjust the number of workers running on each machine. (adjust the number of slots per server)

Supervisor. childopts is used in the storm-deploy project to configure JVM options for the supervisor daemon

Supervisor. Worker. Timeout. secs supervisor's worker heartbeat timeout time. Once the supervisor times out, the supervisor tries to restart the worker process.

Supervisor. Worker. Start. Timeout. secs when the supervisor is started, the worker's heartbeat timeout time. When the time exceeds this time, the supervisor tries to restart the worker. Due to the additional consumption caused by the Initial Startup and configuration of JVM, the first heartbeat will exceed the settings of supervisor. Worker. Timeout. secs.

Supervisor. Enable whether the supervisor should run the workers assigned to him. The default value is true. This option is used for storm unit testing and should not be modified.

Supervisor. Heartbeat. frequency. secs supervisor heartbeat sending frequency (how often)

Supervisor. Monitor. frequency. secs supervisor checks the worker heartbeat frequency

The JVM option used by the worker. childopts supervisor to start the worker. All the "% ID %" strings will be replaced with the corresponding worker identifier

Worker. Heartbeat. frequency. secs interval of Heartbeat sending

Task. Heartbeat. frequency. secs task reporting status heartbeat Interval

Task. refresh. poll. the frequency of link synchronization between secs tasks and other tasks. (if the task is re-allocated, other tasks need to refresh the connection to send messages to it ). generally, other tasks will be notified when the reallocation occurs. This configuration is only used to prevent notifications.

If topology. debug is set to true, storm records each piece of information sent.

Topology. Does optimize master optimize topologies by running multiple tasks in a single thread at the appropriate time.

The number of processes that should be started when topology. Workers executes the topology cluster. Each process internally executes a certain number of tasks in the thread mode. The topology component is combined with the parameter and concurrency prompt to optimize the performance.

Topology. number of Acker tasks started in ackers topology. acker stores the records of tuples sent by spout and detects when tuple is fully processed. when Acker detects that tuple has been processed, it sends a confirmation message to spout. generally, the number of ackers should be determined based on the topology throughput, but generally it does not need too much. when the value is set to 0, message reliability is disabled. Storm will confirm the message immediately after spout sends tuples.

Maximum processing timeout time for spout messages in topology. Message. Timeout. secs topology. If a message is not successfully ack in this time window, storm will notify spout that the message fails. Some spouts enable the failed message replay function.

Topology. kryo. Register registers to the list of serialization schemes of kryo (storm underlying serialization framework). the serialization scheme can be a class name or the implementation of COM. esotericsoftware. kryo. serializer.

Whether topology. Skip. Missing. kryo. registrations storm should skip the unrecognized kryo serialization scheme. If it is set to no task, it may fail to load or throw an error at runtime.

Topology. Max. task. parallelism allows the maximum degree of parallelism of components in a topology. This configuration is mainly used to test the thread count limit in local mode.

Topology. Max. spout. Pending the maximum number of tuples in the pending status of a spout task. This configuration is applied to a single task, rather than the entire spouts or topology.

Topology. state. synchronization. Timeout. Maximum timeout time of the secs component synchronization status Source (retained, not used currently)

Topology. Stats. sample. Rate tuples sampling percentage used to generate task statistics

Whether Java serialization scheme is used in topology. Fall. back. On. java. serialization Topology

Zmq. threads Number of threads used for zeromq communication in each worker process

Zmq. Linger. millis when the connection is closed, the link tries to resend the duration of the message to the target host. This is an uncommon advanced option and can be ignored.

Java. Library. Path: Specifies the java. Library. path when the JVM is started (such as Nimbus, supervisor, and workers). This option tells the JVM where to locate the local library.

Storm [configuration items]-detailed explanation