Kafka Broker Common Configuration detailed

The broker's configuration file is located in the Kafka config/server.properties file.

Broker Basic Configuration

Broker.id: Proxy ID, must be a unique integer. It can be a custom number such as 0,1,2,3, or it can be the last of the IP address, such as 23,24,25, it is recommended to use the following encoding.

Auto.leader.rebalance.enable: If the leader node is allowed to be automatically assigned, if enabled, there will be a background process timed check and trigger the transfer leader operation. Enabled by default.

Background.threads: The number of processes for the background processing task, which defaults to ten.

Topic Configuration

auto.create.topics.enable: Whether the theme is allowed to be created automatically, by default true and recommended to false.

delete.topic.enable: Whether to allow deletion of themes, not allowed by default, recommended to maintain the status quo.

Zookeeper Configuration

Zookeeper.connect:zookeeper the address and port number of the server, separated by commas if there are multiple. such as 10.0.10.1:2181,10.0.10.2:2182.

Zookeeper.connection.timeout.ms:Zookeeper Connection time-out, in MS, default value 6000.

Socket Server Settings

Listeners: Listener server, port number must be consistent with port, plaintext://:9092

Port: Listening ports

Host.name:broker the host address of the binding, either a hostname or an IP address. Default value of localhost. If not set, the server will not be found if you connect through the client.

Advertised.host.name: The port number that the agent broadcasts to producers and consumers, and if not set, the default value is Host.name.

num.network.threads: Number of concurrent network threads, default is 3

Num.io.threads: Number of concurrent IO threads, default = 8

Socket.send.buffer.bytes:socket Send server cache, default 102400 bytes

socket.receive.buffer.bytes:socket receiving server cache, default 102400 bytes

socket.request.max.bytes:socket Maximum number of bytes per request, default 102857600 bytes

Log Configuration

Log.dirs: The Log storage path

num.partitions: The default number of partitions per topic, the default value is 1. The more partitions, the more consumers can be in parallel, but causes the proxy server to process more files.

Num.recovery.threads.per.data.dir: The number of threads per data directory, which is used for flushing when the log is resumed at startup and closed. It is recommended that the data directory in the RAID queue be increased together.

log write to disk policy

Log.flush.interval.messages: The time between two messages written to disk, the default value of 10000, in milliseconds.

log.flush.interval.ms: The maximum time a message occupies a log file before a message is forced to disk.

Log Save Policy

log.retention.hours: Log minimum save time, that is, more than a certain time before it can be deleted, the default value of 168, per hour.

log.retention.bytes: Maximum storage capacity of log files, default 1073741824, Unit bytes.

log.segment.bytes: Maximum storage capacity of a single log file, default 1073741824, Unit bytes.

log.retention.check.interval.ms: The earliest time the log can be deleted, the default is 300000, in milliseconds. That is, the log can be deleted only after 5 minutes of creation.

Broker has a total of 130 configurations, here is not a detailed explanation, interested in the small partner please refer to here.

This article is from the "This person's IT World" blog, be sure to keep this source http://favccxx.blog.51cto.com/2890523/1764716

Kafka Broker Common Configuration detailed