kafka.common.KafkaException: fetching topic metadata for topics

標籤：flume kafka fetching topic metadata for topics

2016-10-26 11:05:29,716  WARN [flume_dx2.zdp.ol-1477451127056-e2015d55-leader-finder-thread] kafka.consumer.ConsumerFetcherManager$LeaderFinderThread (line 89) [flume_dx2.zdp.ol-1477451127056-e2015d55-leader-finder-thread], Failed to find leader for Set([v4-avail-service-request,0])kafka.common.KafkaException: fetching topic metadata for topics [Set(v4-avail-service-request)] from broker [ArrayBuffer(BrokerEndPoint(0,ip-192-168-110-4.cn-north-1.compute.internal,9092))] failedat kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:72)at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:93)at kafka.consumer.ConsumerFetcherManager$LeaderFinderThread.doWork(ConsumerFetcherManager.scala:66)at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:63)Caused by: java.nio.channels.ClosedChannelExceptionat kafka.network.BlockingChannel.send(BlockingChannel.scala:110)at kafka.producer.SyncProducer.liftedTree1$1(SyncProducer.scala:76)at kafka.producer.SyncProducer.kafka$producer$SyncProducer$$doSend(SyncProducer.scala:75)at kafka.producer.SyncProducer.send(SyncProducer.scala:120)at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:58)... 3 more

Flume從Kafka訂閱資料報錯Failed to find leader for Set

Flume部署在本地，Kafka叢集部署在AWS雲上。

解決辦法

Consume端伺服器配置/etc/hosts檔案，其中機器名從要同Kafka服務端私人DNS名稱相同（這個名稱可以在報錯資訊中看到）

123.123.123.123 ip-192-168-110-4.cn-north-1.compute.internal

本文出自 “巧克力黑” 部落格，請務必保留此出處http://10120275.blog.51cto.com/10110275/1865792

kafka.common.KafkaException: fetching topic metadata for topics