Storm's--a detailed description of the configuration

Source: Internet
Author: User
Tags serialization time interval zookeeper

Reprint Please specify source: http://blog.csdn.net/l1028386804/article/details/51924272

Configuration Items

Configuration Instructions

Storm.zookeeper.servers

Zookeeper Server List

Storm.zookeeper.port

Zookeeper Connection Port

Storm.local.dir

Local file system directory used by storm (must exist and the storm process can read and write)

Storm.cluster.mode

Storm cluster run mode ([distributed|local])

Storm.local.mode.zmq

Whether to use ZEROMQ as the message system in local mode, and if set to False, use the Java messaging system. Default is False

Storm.zookeeper.root

Root location of storm in zookeeper

Storm.zookeeper.session.timeout

Client connection Zookeeper time-out

Storm.id

The ID of the running topology, consisting of storm name and a unique random number.

Nimbus.host

Nimbus Server Address

Nimbus.thrift.port

Nimbus Thrift Listening Port

Nimbus.childopts

JVM options assigned to the Nimbus process by Storm-deploy project deployment

Nimbus.task.timeout.secs

The heartbeat timeout, after which the Nimbus considers the task dead and assigns it to another address.

Nimbus.monitor.freq.secs

Nimbus Check the time interval for heartbeat and reassignment tasks. Note If the machine is down, Nimbus will take over and handle it immediately.

Nimbus.supervisor.timeout.secs

Supervisor the heartbeat timeout, once more than Nimbus will think that the supervisor is dead and stop distributing new tasks for it.

Nimbus.task.launch.secs

A special time-out setting when task starts. This value is used to temporarily replace the nimbus.task.timeout.secs before the first heartbeat after startup.

Nimbus.reassign

Nimbus whether to reassign execution when a task is found to fail. The default is true and is not recommended for modification.

Nimbus.file.copy.expiration.secs

Nimbus determines the time-out of the upload/download link, and when the idle time exceeds the setting, Nimbus considers the link dead and actively disconnects

Ui.port

Service ports for the Storm UI

Drpc.servers

DRPC Server list So drpcspout know who to communicate with

Drpc.port

Service port for Storm DRPC

Supervisor.slots.ports

A list of ports on supervisor that can run workers. Each worker occupies one port, and only one worker is running per port. This configuration allows you to adjust the number of workers running on each machine. (Adjust slots/per machine)

Supervisor.childopts

Used in the Storm-deploy project to configure the JVM options for the Supervisor daemon

Supervisor.worker.timeout.secs

The worker heartbeat timeout in supervisor attempts to restart the worker process once the supervisor is timed out.

Supervisor.worker.start.timeout.secs

Supervisor when the worker's heartbeat time-out is initially started, supervisor attempts to restart the worker when it exceeds that time. Because of the additional consumption of the JVM's initial boot and configuration, the first heartbeat will exceed the Supervisor.worker.timeout.secs setting

Supervisor.enable

Supervisor should run the workers assigned to him. The default is true, which is used for unit testing of storm and should not normally be modified.

Supervisor.heartbeat.frequency.secs

Supervisor Heartbeat Send frequency (how often sent)

Supervisor.monitor.frequency.secs

Supervisor Check the rate of the worker's heartbeat

Worker.childopts

Supervisor the JVM option to use when starting the worker. All "%id%" strings are replaced with the identifier of the corresponding worker

Worker.heartbeat.frequency.secs

Heartbeat Send time interval for worker

Task.heartbeat.frequency.secs

Task reporting State heartbeat time interval

Task.refresh.poll.secs

The frequency of link synchronization between a task and other tasks. (If the task is reassigned, other tasks will need to refresh the connection to send it a message). In general, other tasks are understood to be notified when redistribution occurs. This configuration is only intended to prevent non-notification situations.

Topology.debug

If set to True,storm, each message emitted will be recorded.

Topology.optimize

Whether Master is at the right time to optimize topologies by running multiple tasks within a single thread.

Topology.workers

The number of processes that should be started in the topology cluster. Components within each process will execute a certain number of tasks.topology threads in combination with this parameter and the degree of parallelism hint to optimize performance

Topology.ackers

The number of Acker tasks started in topology. Acker save a record of tuples sent by spout, and detects when a tuple is fully processed. When Acker detects that a tuple is processed, it sends a confirmation message to spout. Typically, the number of Acker should be determined based on the throughput of the topology, but it generally does not require much. When set to 0 o'clock, it is equivalent to disabling message reliability. Storm will confirm immediately after the spout sends the tuples.

Topology.message.timeout.secs

Maximum processing time-out for spout messages in topology. If a message is not successful within that time window Ack,storm will tell spout that the message failed. Some spout implement the failed message replay feature.

Topology.kryo.register

A list of serialization schemes registered to Kryo (the storm underlying serialization framework). A serialization scheme can be a class name, or an implementation of Com.esotericsoftware.kryo.Serializer.

Topology.skip.missing.kryo.registrations

Whether Storm should skip the Kryo serialization scheme it does not recognize. If set to no task may mount failed or throw an error at run time.

Topology.max.task.parallelism

The maximum component parallelism that can be allowed in a topology. This configuration is primarily used to test the limit of the number of threads in local mode.

Topology.max.spout.pending

The maximum number of tuples in a spout task that is in the pending state. The configuration applies to a single task instead of the entire spouts or topology.

Topology.state.synchronization.timeout.secs

Maximum timeout time for component synchronization State source (reserved option, not used temporarily)

Topology.stats.sample.rate

Percentage of tuples sampling used to generate task statistics

Topology.fall.back.on.java.serialization

Whether to use the Java serialization scheme in topology

Zmq.threads

Number of threads per worker in-process ZEROMQ communication

Zmq.linger.millis

When the connection is closed, the link attempts to resend the message to the target host for a duration of time. This is an uncommon advanced option that can basically be ignored.

Java.library.path

Java.library.path Settings for JVM startup (such as nimbus,supervisor and workers). This option tells the JVM which paths to locate the local library.


Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: info-contact@alibabacloud.com and provide relevant evidence. A staff member will contact you within 5 working days.

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.