Kafka Message Middleware Learning Notes

1 Key Terms topic the induction of a set of messages. After the producer sends the message to topic, the consumer that subscribes to topic can receive the corresponding message. The messages in the topic are effective and are discarded to free up space after the save period. Consumer the only data that needs to be maintained is the location of the message in the log, which is offset. As consumer continuously reads the message, the offset will also continue to increase. Producer: Sends a message to topic and decides which partition to send it to. Decision Partitioning method: Load balancing mechanism selection and specified by a specific partition function. The second type is usually used. Leader and followers: Each partition is made up of one server as "leader", 0 or several servers as "followers", leader is responsible for processing the read and write of messages, and followers to replicate leader. If leader down. , one of the followers will automatically become leader. Each service in a cluster plays two roles at the same time: as the leader of a portion of the partition it holds, as well as the followers of other partitions, the cluster will have good load balancing. Two methods of message Publishing: Queue mode and publish-subscribe mode. Queue mode: Consumers can read messages from the server at the same time, and each message is read only by one of the consumer. If all of the consumer are in a group, this becomes the traditional queue mode, in which the load-balanced publish-subscribe mode (one-to-many) is implemented in each consumer: the messages in the publish-subscribe mode are broadcast to all the consumer. If all the consumer are not in different groups, this becomes the publish-subscribe mode, and all messages are distributed to all the consumer broker: Represents a Kafka server, When you start Kafka, you need to configure the appropriate server.properties, which has a configuration broker.id=2, which represents a node that is unique in the cluster. Leader: Responsible for processing the read and write of messages, leader is randomly selected from all nodes. Replicas: Lists all replica nodes, regardless of whether the nodes are in the service. ISR: is the node in service. 2 Unique design: 2.1 A distributed, partitioned, replicable messaging system. 2.2 The client and server communicate over the TCP protocol. Kafka provides Java clients and supports multiple languages.
3 for each topic to maintain a partition, the partition is a sequential, unchanging queue, with each message having an ordinal number in the queue. This serial number is maintained by the consumer.
http://blog.csdn.net/smallnest/article/details/38491483
Resources: http://blog.csdn.net/hmsiwtv/article/details/46960053