添加链接
link管理
链接快照平台
  • 输入网页链接,自动生成快照
  • 标签化管理网页链接

存在60个java producer进程,同时生产数据到5个节点的kafka集群。一部分producer丢失数据,一部分producer完全生产不了数据,生产不了数据的producer有一下日志:

09:43:19.354 [kafka-producer-network-thread | producer-1] DEBUG o.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Initialize connection to node vm121:9092 (id: -5 rack: null) for sending metadata request
09:43:19.354 [kafka-producer-network-thread | producer-1] DEBUG o.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Initiating connection to node vm121:9092 (id: -5 rack: null)
09:43:19.370 [kafka-producer-network-thread | producer-1] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name node--5.bytes-sent
09:43:19.371 [kafka-producer-network-thread | producer-1] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name node--5.bytes-received
09:43:19.372 [kafka-producer-network-thread | producer-1] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name node--5.latency
09:43:19.372 [kafka-producer-network-thread | producer-1] DEBUG o.a.kafka.common.network.Selector - [Producer clientId=producer-1] Created socket with SO_RCVBUF = 32768, SO_SNDBUF = 131072, SO_TIMEOUT = 0 to node -5
09:43:19.372 [kafka-producer-network-thread | producer-1] DEBUG o.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Completed connection to node -5. Fetching API versions.
09:43:19.373 [kafka-producer-network-thread | producer-1] DEBUG o.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Initiating API versions fetch from node -5.
09:43:49.397 [kafka-producer-network-thread | producer-1] DEBUG o.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Disconnecting from node -5 due to request timeout.
09:43:49.397 [kafka-producer-network-thread | producer-1] DEBUG o.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available

这个producer所在的服务器telnet 集群每个节点的9092都是可以正常连接的,这个老师能给一点排查意见吗

老师 同步的方式 异常信息

java.util.concurrent.ExecutionException: org.apache.kafka.common.errors.TimeoutException: Failed to update metadata after 60000 ms.
 at org.apache.kafka.clients.producer.KafkaProducer$FutureFailure.(KafkaProducer.java:1150)
 at org.apache.kafka.clients.producer.KafkaProducer.doSend(KafkaProducer.java:846)
 at org.apache.kafka.clients.producer.KafkaProducer.send(KafkaProducer.java:784)
 at org.apache.kafka.clients.producer.KafkaProducer.send(KafkaProducer.java:671)
 at org.cytec.damon.analysis.dataEncryption.EncryptionApp.main(EncryptionApp.java:29)
Caused by: org.apache.kafka.common.errors.TimeoutException: Failed to update metadata after 60000 ms.