Kafka document 1

1. Getting started
   Getting started
   1.1 Introduction
   Introduction
   Kafka is a distributed, partitioned, replicated commit Log Service. It provides the functionality of a messaging system,
   But with a unique design.
   Kafka is a distributed, partitioned, and multi-copy Log Service. It provides many functions of the message system, but has its own unique design.
   What does all that mean?
   What does that mean?
   First let's review some basic messaging terminology:
   First, let's review some basic communication terms:
   1. Kafka maintains feeds of messages in categories called topics.
   Kafka includes topic Functions
   2. We'll call processes that publish messages to a Kafka topic producers.
   Topic messages are generated as producers.
   3. We'll call processes that subscribe to topics and process the feed of published messages consumers ..
   Topic messages are subscribed to as consumers.
   4. Kafka is run as a cluster comprised of one or more servers each of which is called a broker.
   Kafka runs on a cluster.

   So, at a high level, producers send messages over the network to the Kafka cluster which in turn serves them up to consumers like this:

   The producer sends messages and consumers consume messages. The process is like this:

   Communication between the clients and the servers is done with a simple, high-performance, language agnostic TCP protocol. We provide a Java client for Kafka, but clients are available in different ages.

   The client link is the TCP protocol. We provide a Java client, but we support clients in multiple languages.

Kafka contains the concept of a topic. Each procudcer has a topic, and each topic contains multiple partitions. Each partition is ordered internally and the message sequence remains unchanged.
Kafka stores all the data, whether or not the data has been consumed, but the message will be discarded after a certain period of time.
The data mark stone for the consumer is in zookeeper, using an offset control, the consumer can also reset the offset
Each partition has a server serving as the "Leader" and zero or more servers serving as "followers ". The leader processes all requests for reading and writing data to the partition,
And passive followers copy leaders. If the leader fails,
Followers will automatically become new leaders. Each server acts as a partition and a leader following others to balance the load in the cluster.
Producers can select partitions Based on Data keywords.
Kafka consumers provide two modes: queue and publish-subscribe.

Broker configs
Uniqueness of broker. ID identity
Directory of log. dirs messages
Port that accepts the client request
Zookeeper. Connect zookeeper address
Message. Max. bytes maximum message length
Num. Network. threads thread data stream requested by the Network
Num. Io. threads persistent thread count
Queued. Max. Requests maximum Request queue
Host. Name: name of the broker for zookeeper
Advertised. Host. Name for consumers, the producer name
Num. partitions the number of partitions of a topic.
Num. replica. fetchers: How many copies of data are copied, how many followers are in one partition,

Consumer:
Auto. Commit. Enable automatic submission offset
Rebalance. Max. retries: number of times a new consumer joins and tries to reload
Producer:
Request. required. acks: Check whether the cluster is required and whether the partition data has reached the copy.

Kafka document 1