Resetting the offset of the Kafka topic consumer

Source: Internet
Author: User
Tags zookeeper client

If you are using Kafka to distribute messages, there may be exceptions or other errors in the process of data processing that can result in loss or inconsistency. This time you may want to Kafka the data through the new process, we know that Kafka by default will be saved on disk to 7 days of data, you just need to Kafka a topic of the consumer offset to a certain value or a minimum value, You can get the consumer to start spending from the point you set.

Querying the range of topic offset

Use the following command to query to topic:dynamicrange Broker:The minimum value of offset for sparkmaster:9092 :

$/opt/cloudera/parcels/kafka/bin/kafka-run-class. sh kafka.tools.GetOffsetShell--broker-list sparkmaster:9092 -topic dynamicrange-- time- 2

Output

Dynamicrange:0:1288

Query the maximum value of offset:

$/opt/cloudera/parcels/kafka/bin/kafka-run-class. sh kafka.tools.GetOffsetShell--broker-list sparkmaster:9092 -topic dynamicrange-- time- 1

Output

Dynamicrange:0:7885

From the above output can be seen topic:dynamicrange only one partition:0 offset Range:[1288,7885]

Set offset for consumer group

Start Zookeeper Client

$/opt/cloudera/parcels/cdh/lib/zookeeper/bin/zkcli. SH

Set consumer group with the following command:dynamicrangegroup topic:dynamicrange partition:0 with offset of 1288:

set/consumers/dynamicrangegroup/offsets/dynamicrange/0 1288

Reference: Https://metabroadcast.com/blog/resetting-kafka-offsets

Reprint Please specify source: http://www.cnblogs.com/keitsi/p/5685576.html

Resetting the offset of the Kafka topic consumer

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: info-contact@alibabacloud.com and provide relevant evidence. A staff member will contact you within 5 working days.

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.