link管理
链接快照平台
- 输入网页链接，自动生成快照
- 标签化管理网页链接
相关文章推荐
|
玩足球的手套 · java.lang.NoClassDefFo ...· 6 小时前 · |
|
|
乖乖的蜡烛 · View Source· 11 小时前 · |
|
|
帅气的刺猬 · Confluent Platform ...· 11 小时前 · |
|
|
愉快的铁链 · 30个Kafka常见错误小集合-腾讯云开发者 ...· 11 小时前 · |
|
|
气宇轩昂的啄木鸟 · Windows - Clash Verge ...· 2 月前 · |
|
|
豪气的馒头 · 百济神州在2023年欧洲肿瘤内科学会上展示替 ...· 2 月前 · |
|
|
玩足球的乒乓球 · アニメＤＶＤの記事一覧 | マニレビュ - ...· 6 月前 · |
|
|
沉着的脸盆 · 软件： ESPRIT 系列 | Bruker· 11 月前 · |
link管理
›
getting error on starting kafka-docker · Issue #30 · wurstmeister/kafka-docker · GitHub
|
大力的拐杖
3 月前 |
You signed in with another tab or window.
Reload
to refresh your session.
You signed out in another tab or window.
Reload
to refresh your session.
You switched accounts on another tab or window.
Reload
to refresh your session.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement . We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your accountkafka_1 | 2015/02/09 23:26:27 Get http:///var/run/docker.sock/v1.12/containers/78b86f5188b3/json: dial unix /var/run/docker.sock: permission denied
kafka_1 | [2015-02-09 23:26:27,774] FATAL (kafka.Kafka$)
kafka_1 | java.lang.NumberFormatException: For input string: ""
kafka_1 | at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
kafka_1 | at java.lang.Integer.parseInt(Integer.java:493)
kafka_1 | at java.lang.Integer.parseInt(Integer.java:514)
kafka_1 | at scala.collection.immutable.StringLike$class.toInt(StringLike.scala:229)
kafka_1 | at scala.collection.immutable.StringOps.toInt(StringOps.scala:31)
kafka_1 | at kafka.utils.VerifiableProperties.getIntInRange(VerifiableProperties.scala:75)
kafka_1 | at kafka.utils.VerifiableProperties.getIntInRange(VerifiableProperties.scala:48)
kafka_1 | at kafka.server.KafkaConfig.<init>(KafkaConfig.scala:75)
kafka_1 | at kafka.server.KafkaConfig.<init>(KafkaConfig.scala:31)
kafka_1 | at kafka.Kafka$.main(Kafka.scala:35)
kafka_1 | at kafka.Kafka.main(Kafka.scala)
There may be other issues, however it looks like after following directions I can't actually start a kafka instance.
fedora 21, I've installed and am running consul fine so far.
My user is in the docker group and I can run docker info no problem:
➜ kafka-docker git:(master) ✗ docker info
Containers: 7
Images: 80
Storage Driver: devicemapper
Pool Name: docker-253:1-4850042-pool
Pool Blocksize: 65.54 kB
Data file: /var/lib/docker/devicemapper/devicemapper/data
Metadata file: /var/lib/docker/devicemapper/devicemapper/metadata
Data Space Used: 3.126 GB
Data Space Total: 107.4 GB
Metadata Space Used: 4.846 MB
Metadata Space Total: 2.147 GB
Library Version: 1.02.93 (2015-01-30)
Execution Driver: native-0.2
Kernel Version: 3.18.3-201.fc21.x86_64
Operating System: Fedora 21 (Twenty One)
CPUs: 8
Total Memory: 23.5 GiB
Name: gclaptop
ID: TIZJ:SLMZ:OMIP:QWZH:KOKO:M7OJ:M4KC:BCLE:IZSP:CQME:VMBS:L3AN
I just saw some similiar comments about fig not mounting /var/run/docker. And that user upgraded fig and docker. Problem solved.
However I think I'm already on the most recent:
➜ kafka-docker git:(master) ✗ fig --version
fig 1.0.1
➜ kafka-docker git:(master) ✗ docker --version
Docker version 1.4.1, build 5bc2ff8/1.4.1
➜ kafka-docker git:(master) ✗
I did some more work and made some comments in #26
However, given that I set selinux 'permissive' and the original problem went away. I no longer get the permission error.
However now I can't get the sample tests to run:
fig.yml
➜ kafka-docker git:(master) ✗ cat fig.yml
zookeeper:
image: wurstmeister/zookeeper
ports:
- "2181"
kafka:
image: wurstmeister/kafka:0.8.2.0
ports:
- "9092"
links:
- zookeeper:zk
environment:
KAFKA_ADVERTISED_HOST_NAME: 192.168.1.76
volumes:
- /var/run/docker.sock:/var/run/docker.sock
running:
fig up -d
Running the sample test:
➜ kafka-docker git:(master) ✗ fig up -d
Creating kafkadocker_zookeeper_1...
Creating kafkadocker_kafka_1...
➜ kafka-docker git:(master) ✗ ./start-kafka-shell.sh 192.168.1.76 192.168.1.76:2181
root@ffcf92988e44:/# $KAFKA_HOME/bin/kafka-topics.sh --create --topic topic \
> --partitions 4 --zookeeper $ZK --replication-factor 1
[2015-02-10 04:15:03,099] WARN Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect (org.apache.zookeeper.ClientCnxn)
java.net.NoRouteToHostException: No route to host
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:601)
at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)
[2015-02-10 04:15:05,204] WARN Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect (org.apache.zookeeper.ClientCnxn)
java.net.NoRouteToHostException: No route to host
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:601)
at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)
^Croot@ffcf92988e44:/#
fig logs:
➜ kafka-docker git:(master) ✗ fig logs
Attaching to kafkadocker_kafka_1, kafkadocker_zookeeper_1
kafka_1 | [2015-02-10 04:14:43,709] INFO Verifying properties (kafka.utils.VerifiableProperties)
kafka_1 | [2015-02-10 04:14:43,732] INFO Property advertised.host.name is overridden to 192.168.1.76 (kafka.utils.VerifiableProperties)
kafka_1 | [2015-02-10 04:14:43,732] INFO Property advertised.port is overridden to 49163 (kafka.utils.VerifiableProperties)
kafka_1 | [2015-02-10 04:14:43,732] INFO Property broker.id is overridden to 49163 (kafka.utils.VerifiableProperties)
kafka_1 | [2015-02-10 04:14:43,733] INFO Property log.cleaner.enable is overridden to false (kafka.utils.VerifiableProperties)
kafka_1 | [2015-02-10 04:14:43,733] INFO Property log.dirs is overridden to /kafka/kafka-logs-49163 (kafka.utils.VerifiableProperties)
kafka_1 | [2015-02-10 04:14:43,733] INFO Property log.retention.check.interval.ms is overridden to 300000 (kafka.utils.VerifiableProperties)
kafka_1 | [2015-02-10 04:14:43,733] INFO Property log.retention.hours is overridden to 168 (kafka.utils.VerifiableProperties)
kafka_1 | [2015-02-10 04:14:43,733] INFO Property log.segment.bytes is overridden to 1073741824 (kafka.utils.VerifiableProperties)
kafka_1 | [2015-02-10 04:14:43,733] INFO Property num.io.threads is overridden to 8 (kafka.utils.VerifiableProperties)
kafka_1 | [2015-02-10 04:14:43,733] INFO Property num.network.threads is overridden to 3 (kafka.utils.VerifiableProperties)
kafka_1 | [2015-02-10 04:14:43,733] INFO Property num.partitions is overridden to 1 (kafka.utils.VerifiableProperties)
kafka_1 | [2015-02-10 04:14:43,734] INFO Property num.recovery.threads.per.data.dir is overridden to 1 (kafka.utils.VerifiableProperties)
kafka_1 | [2015-02-10 04:14:43,734] INFO Property port is overridden to 9092 (kafka.utils.VerifiableProperties)
kafka_1 | [2015-02-10 04:14:43,734] INFO Property socket.receive.buffer.bytes is overridden to 102400 (kafka.utils.VerifiableProperties)
kafka_1 | [2015-02-10 04:14:43,734] INFO Property socket.request.max.bytes is overridden to 104857600 (kafka.utils.VerifiableProperties)
kafka_1 | [2015-02-10 04:14:43,734] INFO Property socket.send.buffer.bytes is overridden to 102400 (kafka.utils.VerifiableProperties)
kafka_1 | [2015-02-10 04:14:43,734] INFO Property zookeeper.connect is overridden to 172.17.0.14:2181 (kafka.utils.VerifiableProperties)
kafka_1 | [2015-02-10 04:14:43,735] INFO Property zookeeper.connection.timeout.ms is overridden to 6000 (kafka.utils.VerifiableProperties)
kafka_1 | [2015-02-10 04:14:43,759] INFO [Kafka Server 49163], starting (kafka.server.KafkaServer)
kafka_1 | [2015-02-10 04:14:43,761] INFO [Kafka Server 49163], Connecting to zookeeper on 172.17.0.14:2181 (kafka.server.KafkaServer)
kafka_1 | [2015-02-10 04:14:43,767] INFO Starting ZkClient event thread. (org.I0Itec.zkclient.ZkEventThread)
kafka_1 | [2015-02-10 04:14:43,772] INFO Client environment:zookeeper.version=3.4.6-1569965, built on 02/20/2014 09:09 GMT (org.apache.zookeeper.ZooKeeper)
kafka_1 | [2015-02-10 04:14:43,772] INFO Client environment:host.name=78fee06e17e8 (org.apache.zookeeper.ZooKeeper)
kafka_1 | [2015-02-10 04:14:43,772] INFO Client environment:java.version=1.6.0_34 (org.apache.zookeeper.ZooKeeper)
kafka_1 | [2015-02-10 04:14:43,772] INFO Client environment:java.vendor=Sun Microsystems Inc. (org.apache.zookeeper.ZooKeeper)
kafka_1 | [2015-02-10 04:14:43,772] INFO Client environment:java.home=/usr/lib/jvm/java-6-openjdk-amd64/jre (org.apache.zookeeper.ZooKeeper)
kafka_1 | [2015-02-10 04:14:43,772] INFO Client environment:java.class.path=:/opt/kafka_2.10-0.8.2.0/bin/../core/build/dependant-libs-2.10.4*/*.jar:/opt/kafka_2.10-0.8.2.0/bin/../examples/build/libs//kafka-examples*.jar:/opt/kafka_2.10-0.8.2.0/bin/../contrib/hadoop-consumer/build/libs//kafka-hadoop-consumer*.jar:/opt/kafka_2.10-0.8.2.0/bin/../contrib/hadoop-producer/build/libs//kafka-hadoop-producer*.jar:/opt/kafka_2.10-0.8.2.0/bin/../clients/build/libs/kafka-clients*.jar:/opt/kafka_2.10-0.8.2.0/bin/../libs/jopt-simple-3.2.jar:/opt/kafka_2.10-0.8.2.0/bin/../libs/kafka-clients-0.8.2.0.jar:/opt/kafka_2.10-0.8.2.0/bin/../libs/kafka_2.10-0.8.2.0-javadoc.jar:/opt/kafka_2.10-0.8.2.0/bin/../libs/kafka_2.10-0.8.2.0-scaladoc.jar:/opt/kafka_2.10-0.8.2.0/bin/../libs/kafka_2.10-0.8.2.0-sources.jar:/opt/kafka_2.10-0.8.2.0/bin/../libs/kafka_2.10-0.8.2.0-test.jar:/opt/kafka_2.10-0.8.2.0/bin/../libs/kafka_2.10-0.8.2.0.jar:/opt/kafka_2.10-0.8.2.0/bin/../libs/log4j-1.2.16.jar:/opt/kafka_2.10-0.8.2.0/bin/../libs/lz4-1.2.0.jar:/opt/kafka_2.10-0.8.2.0/bin/../libs/metrics-core-2.2.0.jar:/opt/kafka_2.10-0.8.2.0/bin/../libs/scala-library-2.10.4.jar:/opt/kafka_2.10-0.8.2.0/bin/../libs/slf4j-api-1.7.6.jar:/opt/kafka_2.10-0.8.2.0/bin/../libs/slf4j-log4j12-1.6.1.jar:/opt/kafka_2.10-0.8.2.0/bin/../libs/snappy-java-1.1.1.6.jar:/opt/kafka_2.10-0.8.2.0/bin/../libs/zkclient-0.3.jar:/opt/kafka_2.10-0.8.2.0/bin/../libs/zookeeper-3.4.6.jar:/opt/kafka_2.10-0.8.2.0/bin/../core/build/libs/kafka_2.10*.jar (org.apache.zookeeper.ZooKeeper)
kafka_1 | [2015-02-10 04:14:43,772] INFO Client environment:java.library.path=/usr/lib/jvm/java-6-openjdk-amd64/jre/lib/amd64/server:/usr/lib/jvm/java-6-openjdk-amd64/jre/lib/amd64:/usr/lib/jvm/java-6-openjdk-amd64/jre/../lib/amd64:/usr/java/packages/lib/amd64:/usr/lib/x86_64-linux-gnu/jni:/lib/x86_64-linux-gnu:/usr/lib/x86_64-linux-gnu:/usr/lib/jni:/lib:/usr/lib (org.apache.zookeeper.ZooKeeper)
kafka_1 | [2015-02-10 04:14:43,772] INFO Client environment:java.io.tmpdir=/tmp (org.apache.zookeeper.ZooKeeper)
kafka_1 | [2015-02-10 04:14:43,772] INFO Client environment:java.compiler=<NA> (org.apache.zookeeper.ZooKeeper)
kafka_1 | [2015-02-10 04:14:43,772] INFO Client environment:os.name=Linux (org.apache.zookeeper.ZooKeeper)
kafka_1 | [2015-02-10 04:14:43,772] INFO Client environment:os.arch=amd64 (org.apache.zookeeper.ZooKeeper)
kafka_1 | [2015-02-10 04:14:43,772] INFO Client environment:os.version=3.18.5-201.fc21.x86_64 (org.apache.zookeeper.ZooKeeper)
kafka_1 | [2015-02-10 04:14:43,772] INFO Client environment:user.name=root (org.apache.zookeeper.ZooKeeper)
kafka_1 | [2015-02-10 04:14:43,773] INFO Client environment:user.home=/root (org.apache.zookeeper.ZooKeeper)
kafka_1 | [2015-02-10 04:14:43,773] INFO Client environment:user.dir=/ (org.apache.zookeeper.ZooKeeper)
kafka_1 | [2015-02-10 04:14:43,773] INFO Initiating client connection, connectString=172.17.0.14:2181 sessionTimeout=6000 watcher=org.I0Itec.zkclient.ZkClient@4b704006 (org.apache.zookeeper.ZooKeeper)
kafka_1 | [2015-02-10 04:14:43,792] INFO Opening socket connection to server 172.17.0.14/172.17.0.14:2181. Will not attempt to authenticate using SASL (unknown error) (org.apache.zookeeper.ClientCnxn)
kafka_1 | [2015-02-10 04:14:43,795] INFO Socket connection established to 172.17.0.14/172.17.0.14:2181, initiating session (org.apache.zookeeper.ClientCnxn)
kafka_1 | [2015-02-10 04:14:43,834] INFO Session establishment complete on server 172.17.0.14/172.17.0.14:2181, sessionid = 0x14b71b243190000, negotiated timeout = 6000 (org.apache.zookeeper.ClientCnxn)
kafka_1 | [2015-02-10 04:14:43,835] INFO zookeeper state changed (SyncConnected) (org.I0Itec.zkclient.ZkClient)
kafka_1 | [2015-02-10 04:14:44,061] INFO Log directory '/kafka/kafka-logs-49163' not found, creating it. (kafka.log.LogManager)
kafka_1 | [2015-02-10 04:14:44,068] INFO Loading logs. (kafka.log.LogManager)
kafka_1 | [2015-02-10 04:14:44,072] INFO Logs loading complete. (kafka.log.LogManager)
kafka_1 | [2015-02-10 04:14:44,072] INFO Starting log cleanup with a period of 300000 ms. (kafka.log.LogManager)
kafka_1 | [2015-02-10 04:14:44,075] INFO Starting log flusher with a default period of 9223372036854775807 ms. (kafka.log.LogManager)
kafka_1 | [2015-02-10 04:14:44,095] INFO Awaiting socket connections on 0.0.0.0:9092. (kafka.network.Acceptor)
kafka_1 | [2015-02-10 04:14:44,095] INFO [Socket Server on Broker 49163], Started (kafka.network.SocketServer)
kafka_1 | [2015-02-10 04:14:44,140] INFO Will not load MX4J, mx4j-tools.jar is not in the classpath (kafka.utils.Mx4jLoader$)
kafka_1 | [2015-02-10 04:14:44,172] INFO 49163 successfully elected as leader (kafka.server.ZookeeperLeaderElector)
kafka_1 | [2015-02-10 04:14:44,260] INFO Registered broker 49163 at path /brokers/ids/49163 with address 192.168.1.76:49163. (kafka.utils.ZkUtils$)
kafka_1 | [2015-02-10 04:14:44,271] INFO [Kafka Server 49163], started (kafka.server.KafkaServer)
kafka_1 | [2015-02-10 04:14:44,303] INFO New leader is 49163 (kafka.server.ZookeeperLeaderElector$LeaderChangeListener)
zookeeper_1 | JMX enabled by default
zookeeper_1 | Using config: /opt/zookeeper-3.4.6/bin/../conf/zoo.cfg
zookeeper_1 | 2015-02-10 04:14:42,835 [myid:] - INFO [main:QuorumPeerConfig@103] - Reading configuration from: /opt/zookeeper-3.4.6/bin/../conf/zoo.cfg
zookeeper_1 | 2015-02-10 04:14:42,910 [myid:] - INFO [main:DatadirCleanupManager@78] - autopurge.snapRetainCount set to 3
zookeeper_1 | 2015-02-10 04:14:42,910 [myid:] - INFO [main:DatadirCleanupManager@79] - autopurge.purgeInterval set to 1
zookeeper_1 | 2015-02-10 04:14:42,911 [myid:] - WARN [main:QuorumPeerMain@113] - Either no config or no quorum defined in config, running in standalone mode
zookeeper_1 | 2015-02-10 04:14:42,911 [myid:] - INFO [PurgeTask:DatadirCleanupManager$PurgeTask@138] - Purge task started.
zookeeper_1 | 2015-02-10 04:14:42,936 [myid:] - INFO [PurgeTask:DatadirCleanupManager$PurgeTask@144] - Purge task completed.
zookeeper_1 | 2015-02-10 04:14:42,939 [myid:] - INFO [main:QuorumPeerConfig@103] - Reading configuration from: /opt/zookeeper-3.4.6/bin/../conf/zoo.cfg
zookeeper_1 | 2015-02-10 04:14:42,940 [myid:] - INFO [main:ZooKeeperServerMain@95] - Starting server
zookeeper_1 | 2015-02-10 04:14:42,945 [myid:] - INFO [main:Environment@100] - Server environment:zookeeper.version=3.4.6-1569965, built on 02/20/2014 09:09 GMT
zookeeper_1 | 2015-02-10 04:14:42,945 [myid:] - INFO [main:Environment@100] - Server environment:host.name=2eab46a0e449
zookeeper_1 | 2015-02-10 04:14:42,945 [myid:] - INFO [main:Environment@100] - Server environment:java.version=1.7.0_65
zookeeper_1 | 2015-02-10 04:14:42,945 [myid:] - INFO [main:Environment@100] - Server environment:java.vendor=Oracle Corporation
zookeeper_1 | 2015-02-10 04:14:42,945 [myid:] - INFO [main:Environment@100] - Server environment:java.home=/usr/lib/jvm/java-7-openjdk-amd64/jre
zookeeper_1 | 2015-02-10 04:14:42,945 [myid:] - INFO [main:Environment@100] - Server environment:java.class.path=/opt/zookeeper-3.4.6/bin/../build/classes:/opt/zookeeper-3.4.6/bin/../build/lib/*.jar:/opt/zookeeper-3.4.6/bin/../lib/slf4j-log4j12-1.6.1.jar:/opt/zookeeper-3.4.6/bin/../lib/slf4j-api-1.6.1.jar:/opt/zookeeper-3.4.6/bin/../lib/netty-3.7.0.Final.jar:/opt/zookeeper-3.4.6/bin/../lib/log4j-1.2.16.jar:/opt/zookeeper-3.4.6/bin/../lib/jline-0.9.94.jar:/opt/zookeeper-3.4.6/bin/../zookeeper-3.4.6.jar:/opt/zookeeper-3.4.6/bin/../src/java/lib/*.jar:/opt/zookeeper-3.4.6/bin/../conf:
zookeeper_1 | 2015-02-10 04:14:42,945 [myid:] - INFO [main:Environment@100] - Server environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib/x86_64-linux-gnu/jni:/lib/x86_64-linux-gnu:/usr/lib/x86_64-linux-gnu:/usr/lib/jni:/lib:/usr/lib
zookeeper_1 | 2015-02-10 04:14:42,946 [myid:] - INFO [main:Environment@100] - Server environment:java.io.tmpdir=/tmp
zookeeper_1 | 2015-02-10 04:14:42,947 [myid:] - INFO [main:Environment@100] - Server environment:java.compiler=<NA>
zookeeper_1 | 2015-02-10 04:14:42,947 [myid:] - INFO [main:Environment@100] - Server environment:os.name=Linux
zookeeper_1 | 2015-02-10 04:14:42,947 [myid:] - INFO [main:Environment@100] - Server environment:os.arch=amd64
zookeeper_1 | 2015-02-10 04:14:42,948 [myid:] - INFO [main:Environment@100] - Server environment:os.version=3.18.5-201.fc21.x86_64
zookeeper_1 | 2015-02-10 04:14:42,948 [myid:] - INFO [main:Environment@100] - Server environment:user.name=root
zookeeper_1 | 2015-02-10 04:14:42,948 [myid:] - INFO [main:Environment@100] - Server environment:user.home=/root
zookeeper_1 | 2015-02-10 04:14:42,948 [myid:] - INFO [main:Environment@100] - Server environment:user.dir=/opt/zookeeper-3.4.6
zookeeper_1 | 2015-02-10 04:14:42,948 [myid:] - INFO [main:ZooKeeperServer@755] - tickTime set to 2000
zookeeper_1 | 2015-02-10 04:14:42,949 [myid:] - INFO [main:ZooKeeperServer@764] - minSessionTimeout set to -1
zookeeper_1 | 2015-02-10 04:14:42,949 [myid:] - INFO [main:ZooKeeperServer@773] - maxSessionTimeout set to -1
zookeeper_1 | 2015-02-10 04:14:42,959 [myid:] - INFO [main:NIOServerCnxnFactory@94] - binding to port 0.0.0.0/0.0.0.0:2181
zookeeper_1 | 2015-02-10 04:14:43,797 [myid:] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxnFactory@197] - Accepted socket connection from /172.17.0.15:37961
zookeeper_1 | 2015-02-10 04:14:43,802 [myid:] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:ZooKeeperServer@868] - Client attempting to establish new session at /172.17.0.15:37961
zookeeper_1 | 2015-02-10 04:14:43,804 [myid:] - INFO [SyncThread:0:FileTxnLog@199] - Creating new log file: log.1
zookeeper_1 | 2015-02-10 04:14:43,833 [myid:] - INFO [SyncThread:0:ZooKeeperServer@617] - Established session 0x14b71b243190000 with negotiated timeout 6000 for client /172.17.0.15:37961
zookeeper_1 | 2015-02-10 04:14:43,867 [myid:] - INFO [ProcessThread(sid:0 cport:-1)::PrepRequestProcessor@645] - Got user-level KeeperException when processing sessionid:0x14b71b243190000 type:create cxid:0x4 zxid:0x3 txntype:-1 reqpath:n/a Error Path:/brokers Error:KeeperErrorCode = NoNode for /brokers
zookeeper_1 | 2015-02-10 04:14:43,918 [myid:] - INFO [ProcessThread(sid:0 cport:-1)::PrepRequestProcessor@645] - Got user-level KeeperException when processing sessionid:0x14b71b243190000 type:create cxid:0xa zxid:0x7 txntype:-1 reqpath:n/a Error Path:/config Error:KeeperErrorCode = NoNode for /config
zookeeper_1 | 2015-02-10 04:14:43,964 [myid:] - INFO [ProcessThread(sid:0 cport:-1)::PrepRequestProcessor@645] - Got user-level KeeperException when processing sessionid:0x14b71b243190000 type:create cxid:0x10 zxid:0xb txntype:-1 reqpath:n/a Error Path:/admin Error:KeeperErrorCode = NoNode for /admin
zookeeper_1 | 2015-02-10 04:14:44,175 [myid:] - INFO [ProcessThread(sid:0 cport:-1)::PrepRequestProcessor@645] - Got user-level KeeperException when processing sessionid:0x14b71b243190000 type:setData cxid:0x1a zxid:0xf txntype:-1 reqpath:n/a Error Path:/controller_epoch Error:KeeperErrorCode = NoNode for /controller_epoch
zookeeper_1 | 2015-02-10 04:14:44,227 [myid:] - INFO [ProcessThread(sid:0 cport:-1)::PrepRequestProcessor@645] - Got user-level KeeperException when processing sessionid:0x14b71b243190000 type:delete cxid:0x27 zxid:0x11 txntype:-1 reqpath:n/a Error Path:/admin/preferred_replica_election Error:KeeperErrorCode = NoNode for /admin/preferred_replica_election
I only see the 'INFO' messages above, I didn't see anything obvious.
Still nothing here @wurstmeister I haven't been able to get anything working. Following the instructions on the wiki and on the github page.
Any suggestions would be appreciated.
192.168.1.76 is 'localhost'
machine network config
docker0: flags=4099<UP,BROADCAST,MULTICAST> mtu 1500
inet 172.17.42.1 netmask 255.255.0.0 broadcast 0.0.0.0
inet6 fe80::5484:7aff:fefe:9799 prefixlen 64 scopeid 0x20<link>
ether 56:84:7a:fe:97:99 txqueuelen 0 (Ethernet)
RX packets 223 bytes 13772 (13.4 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 1137 bytes 188537 (184.1 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
wls1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 192.168.1.76 netmask 255.255.255.0 broadcast 192.168.1.255
inet6 fe80::2ab2:bdff:feef:bb18 prefixlen 64 scopeid 0x20<link>
ether 28:b2:bd:ef:bb:18 txqueuelen 1000 (Ethernet)
RX packets 340937 bytes 417854923 (398.4 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 202525 bytes 32628116 (31.1 MiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
fig.yml
➜ kafka-docker git:(master) ✗ cat fig.yml
zookeeper:
image: wurstmeister/zookeeper
ports:
- "2181"
kafka:
image: wurstmeister/kafka:0.8.2.0
ports:
- "9092"
links:
- zookeeper:zk
environment:
KAFKA_ADVERTISED_HOST_NAME: 192.168.1.76
volumes:
- /var/run/docker.sock:/var/run/docker.sock
running fig up -d
➜ kafka-docker git:(master) ✗ fig ps
Name Command State Ports
------------------------------
➜ kafka-docker git:(master) ✗ fig up -d
Creating kafkadocker_zookeeper_1...
Creating kafkadocker_kafka_1...
➜ kafka-docker git:(master) ✗ fig ps
Name Command State Ports
-----------------------------------------------------------------------------------------
kafkadocker_kafka_1 /bin/sh -c start- Up 0.0.0.0:49170->9092/
kafka.sh tcp
kafkadocker_zookeepe /bin/sh -c Up 0.0.0.0:49169->2181/
r_1 /usr/sbin/sshd ... tcp, 22/tcp,
2888/tcp, 3888/tcp
➜ kafka-docker git:(master) ✗
In theory I should now be good to go on localhost:2181 / 9092 right?
➜ kafka-docker git:(master) ✗ ./start-kafka-shell.sh 192.168.1.76 192.168.1.76:2181
root@5422d75c46c8:/# $KAFKA_HOME/bin/kafka-topics.sh --create --topic topic \
> --partitions 1 --zookeeper $ZK --replication-factor 1
[2015-02-20 04:22:56,032] WARN Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect (org.apache.zookeeper.ClientCnxn)
java.net.NoRouteToHostException: No route to host
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:601)
at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)
^Croot@5422d75c46c8:/#
And running the console against what should be on localhost:
➜ kafka_2.9.2-0.8.2.0 bin/kafka-topics.sh --list --zookeeper 192.168.1.76:2181
[2015-02-19 20:24:19,094] WARN Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect (org.apache.zookeeper.ClientCnxn)
java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739)
at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)
[2015-02-19 20:24:20,198] WARN Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect (org.apache.zookeeper.ClientCnxn)
java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739)
at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)
I've updated my docker version:
➜ kafka_2.9.2-0.8.2.0 docker version
Client version: 1.5.0
Client API version: 1.17
Go version (client): go1.3.3
Git commit (client): a8a31ef/1.5.0
OS/Arch (client): linux/amd64
Server version: 1.5.0
Server API version: 1.17
Go version (server): go1.3.3
Git commit (server): a8a31ef/1.5.0
This is sample code that works when talking to a local kafka instance:
➜ LegacyProducer cat sarama.debug.log
PREFIX: 2015/02/19 20:31:39 client.go:42: Initializing new client
PREFIX: 2015/02/19 20:31:39 client.go:353: Fetching metadata for [] from broker localhost:9092
PREFIX: 2015/02/19 20:31:39 broker.go:104: Failed to connect to broker localhost:9092: dial tcp 127.0.0.1:9092: connection refused
PREFIX: 2015/02/19 20:31:39 client.go:377: Error from broker while fetching metadata: dial tcp 127.0.0.1:9092: connection refused
PREFIX: 2015/02/19 20:31:39 client.go:310: Disconnecting Broker -1
PREFIX: 2015/02/19 20:31:39 client.go:382: Out of available brokers.
PREFIX: 2015/02/19 20:31:39 client.go:385: Resurrecting dead brokers after 250ms... (3 retries remaining)
PREFIX: 2015/02/19 20:31:39 broker.go:135: Failed to close connection to broker localhost:9092: kafka: broker: not connected
Ok, found part of the problem, I'll call this poor instructions. 9092 isn't the local exposed port. Simple enough, however:
with:
➜ kafka-docker git:(master) ✗ fig ps
Name Command State Ports
-----------------------------------------------------------------------------------------
kafkadocker_kafka_1 /bin/sh -c start- Up 0.0.0.0:49174->9092/
kafka.sh tcp
kafkadocker_kafka_2 /bin/sh -c start- Up 0.0.0.0:49175->9092/
kafka.sh tcp
kafkadocker_kafka_3 /bin/sh -c start- Up 0.0.0.0:49176->9092/
kafka.sh tcp
kafkadocker_zookeepe /bin/sh -c Up 0.0.0.0:49173->2181/
r_1 /usr/sbin/sshd ... tcp, 22/tcp,
2888/tcp, 3888/tcp
I get:
➜ kafka_2.9.2-0.8.2.0 bin/kafka-console-producer.sh --broker-list localhost:49174 --topic my_topic
[2015-02-19 21:14:10,476] WARN Property topic is not valid (kafka.utils.VerifiableProperties)
[2015-02-19 21:14:12,621] WARN Error while fetching metadata [{TopicMetadata for topic my_topic ->
No partition metadata for topic my_topic due to kafka.common.LeaderNotAvailableException}] for topic [my_topic]: class kafka.common.LeaderNotAvailableException (kafka.producer.BrokerPartitionInfo)
[2015-02-19 21:14:12,629] WARN Error while fetching metadata [{TopicMetadata for topic my_topic ->
No partition metadata for topic my_topic due to kafka.common.LeaderNotAvailableException}] for topic [my_topic]: class kafka.common.LeaderNotAvailableException (kafka.producer.BrokerPartitionInfo)
[2015-02-19 21:14:12,630] ERROR Failed to collate messages by topic, partition due to: Failed to fetch topic metadata for topic: my_topic (kafka.producer.async.DefaultEventHandler)
[2015-02-19 21:14:12,737] WARN Error while fetching metadata [{TopicMetadata for topic my_topic ->
No partition metadata for topic my_topic due to kafka.common.LeaderNotAvailableException}] for topic [my_topic]: class kafka.common.LeaderNotAvailableException (kafka.producer.BrokerPartitionInfo)
[2015-02-19 21:14:12,741] WARN Error while fetching metadata [{TopicMetadata for topic my_topic ->
Is it possible that the kafka client can't talk to the zookeeper instance?
it looks like you are experiencing the issue described in #27
I’ve added an example of a fig file with fixed ports for a single broker setup.
fig -f fig-single-broker.yml up
make sure you run fig rm before to avoid any issues with existing containers.
Please let me know if this solves your problem.
Thanks
@wurstmeister thank you for taking a look, however this doesn't solve the 'class kafka.common.LeaderNotAvailableException' problem.
Not sure where that is happening, I'm wondering if the networking between the zookeeper instance and the kafka instance is falling apart there.
I was able to get my own kafka docker thing going in the simplest possible way (After adding an element to my hosts file to handle the client lookup):
https://github.com/luck02/dkafka single zookeeper instance / kafka instance on the same container + a hosts entry that is NOT in that repo.
In any case I'll take a closer look at this tonite and see if I can deduce where the failure is happening.
Just to confirm, did you try to use https://github.com/wurstmeister/kafka-docker/blob/master/fig-single-broker.yml after running fig rm?
When you get a chance, could you please share the output of
$KAFKA_HOME/bin/kafka-topics.sh --describe --zookeeper $ZK
and fig ps or docker ps
also from #30 (comment) to #30 (comment) you increased the number of brokers and the port numbers changed (because of #27). Did you re-create the topic after you restarted the brokers?
Above was from my work laptop which I don't have with me. This is from a xubuntu 14.04 VM on my local workstation:
after running fig stop / fig rm:
➜ kafka-docker git:(master) ✗ cat fig-single-broker.yml
zookeeper:
image: wurstmeister/zookeeper
ports:
- "2181:2181"
kafka:
image: wurstmeister/kafka:0.8.2.0
ports:
- "9092:9092"
links:
- zookeeper:zk
environment:
KAFKA_ADVERTISED_HOST_NAME: 10.103.13.160
volumes:
- /var/run/docker.sock:/var/run/docker.sock
IP is from my eth0 connection
➜ kafka-docker git:(master) ✗ fig -f fig-single-broker.yml up -d
Recreating kafkadocker_zookeeper_1...
Recreating kafkadocker_kafka_1...
Moving to a local kafka directory to test with the kafka console clients:
➜ kafka_2.9.2-0.8.2.0 bin/kafka-topics.sh --list --zookeeper localhost:2181
➜ kafka_2.9.2-0.8.2.0 bin/kafka-topics.sh --list --zookeeper localhost:2181
➜ kafka_2.9.2-0.8.2.0 bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic test
Created topic "test".
➜ kafka_2.9.2-0.8.2.0 bin/kafka-topics.sh --list --zookeeper localhost:2181
➜ kafka_2.9.2-0.8.2.0 bin/kafka-console-producer.sh --broker-list localhost:9092 --topic test
[2015-02-20 08:52:27,598] WARN Property topic is not valid (kafka.utils.VerifiableProperties)
First Test
client test
Other terminal window:
➜ kafka_2.9.2-0.8.2.0 bin/kafka-console-consumer.sh --zookeeper localhost:2181 --topic test --from-beginning
First Test
client test
So that certainly seems to work, I will attempt at home again this evening on my laptop.
Thank you, much appreciated.
Ok, so on my laptop (Fedora 21)
➜ kafka-docker git:(master) ✗ fig rm
Going to remove kafkadocker_kafka_1, kafkadocker_zookeeper_1
Are you sure? [yN]
Removing kafkadocker_zookeeper_1...
Removing kafkadocker_kafka_1...
➜ kafka-docker git:(master) ✗ fig -f fig-single-broker.yml up -d
Creating kafkadocker_zookeeper_1...
Creating kafkadocker_kafka_1...
➜ kafka-docker git:(master) ✗
So we should have a completely clean instance now.
➜ kafka_2.9.2-0.8.2.0 bin/kafka-topics.sh --list --zookeeper 192.168.1.76:2181
➜ kafka_2.9.2-0.8.2.0 bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic test
Created topic "test".
➜ kafka_2.9.2-0.8.2.0 bin/kafka-topics.sh --list --zookeeper 192.168.1.76:2181
➜ kafka_2.9.2-0.8.2.0 bin/kafka-console-producer.sh --broker-list localhost:9092 --topic test
[2015-02-20 20:03:28,509] WARN Property topic is not valid (kafka.utils.VerifiableProperties)
[2015-02-20 20:03:30,680] WARN Error while fetching metadata [{TopicMetadata for topic test ->
No partition metadata for topic test due to kafka.common.LeaderNotAvailableException}] for topic [test]: class kafka.common.LeaderNotAvailableException (kafka.producer.BrokerPartitionInfo)
[2015-02-20 20:03:30,687] WARN Error while fetching metadata [{TopicMetadata for topic test ->
No partition metadata for topic test due to kafka.common.LeaderNotAvailableException}] for topic [test]: class kafka.common.LeaderNotAvailableException (kafka.producer.BrokerPartitionInfo)
[2015-02-20 20:03:30,688] ERROR Failed to collate messages by topic, partition due to: Failed to fetch topic metadata for topic: test (kafka.producer.async.DefaultEventHandler)
[2015-02-20 20:03:30,794] WARN Error while fetching metadata [{TopicMetadata for topic test ->
No partition metadata for topic test due to kafka.common.LeaderNotAvailableException}] for topic [test]: class kafka.common.LeaderNotAvailableException (kafka.producer.BrokerPartitionInfo)
[2015-02-20 20:03:30,798] WARN Error while fetching metadata [{TopicMetadata for topic test ->
No partition metadata for topic test due to kafka.common.LeaderNotAvailableException}] for topic [test]: class kafka.common.LeaderNotAvailableException (kafka.producer.BrokerPartitionInfo)
[2015-02-20 20:03:30,798] ERROR Failed to collate messages by topic, partition due to: Failed to fetch topic metadata for topic: test (kafka.producer.async.DefaultEventHandler)
[2015-02-20 20:03:30,903] WARN Error while fetching metadata [{TopicMetadata for topic test ->
No partition metadata for topic test due to kafka.common.LeaderNotAvailableException}] for topic [test]: class kafka.common.LeaderNotAvailableException (kafka.producer.BrokerPartitionInfo)
[2015-02-20 20:03:30,907] WARN Error while fetching metadata [{TopicMetadata for topic test ->
No partition metadata for topic test due to kafka.common.LeaderNotAvailableException}] for topic [test]: class kafka.common.LeaderNotAvailableException (kafka.producer.BrokerPartitionInfo)
[2015-02-20 20:03:30,907] ERROR Failed to collate messages by topic, partition due to: Failed to fetch topic metadata for topic: test (kafka.producer.async.DefaultEventHandler)
[2015-02-20 20:03:31,013] WARN Error while fetching metadata [{TopicMetadata for topic test ->
No partition metadata for topic test due to kafka.common.LeaderNotAvailableException}] for topic [test]: class kafka.common.LeaderNotAvailableException (kafka.producer.BrokerPartitionInfo)
[2015-02-20 20:03:31,018] WARN Error while fetching metadata [{TopicMetadata for topic test ->
No partition metadata for topic test due to kafka.common.LeaderNotAvailableException}] for topic [test]: class kafka.common.LeaderNotAvailableException (kafka.producer.BrokerPartitionInfo)
[2015-02-20 20:03:31,018] ERROR Failed to collate messages by topic, partition due to: Failed to fetch topic metadata for topic: test (kafka.producer.async.DefaultEventHandler)
[2015-02-20 20:03:31,123] WARN Error while fetching metadata [{TopicMetadata for topic test ->
No partition metadata for topic test due to kafka.common.LeaderNotAvailableException}] for topic [test]: class kafka.common.LeaderNotAvailableException (kafka.producer.BrokerPartitionInfo)
[2015-02-20 20:03:31,124] ERROR Failed to send requests for topics test with correlation ids in [0,8] (kafka.producer.async.DefaultEventHandler)
[2015-02-20 20:03:31,125] ERROR Error in handling batch of 1 events (kafka.producer.async.ProducerSendThread)
kafka.common.FailedToSendMessageException: Failed to send messages after 3 tries.
at kafka.producer.async.DefaultEventHandler.handle(DefaultEventHandler.scala:90)
at kafka.producer.async.ProducerSendThread.tryToHandle(ProducerSendThread.scala:105)
at kafka.producer.async.ProducerSendThread$$anonfun$processEvents$3.apply(ProducerSendThread.scala:88)
at kafka.producer.async.ProducerSendThread$$anonfun$processEvents$3.apply(ProducerSendThread.scala:68)
at scala.collection.immutable.Stream.foreach(Stream.scala:526)
at kafka.producer.async.ProducerSendThread.processEvents(ProducerSendThread.scala:67)
at kafka.producer.async.ProducerSendThread.run(ProducerSendThread.scala:45)
^C% ➜ kafka_2.9.2-0.8.2.0
So on fedora the no leader thing remains.
output you requested:
➜ kafka_2.9.2-0.8.2.0 ./bin/kafka-topics.sh --describe --zookeeper localhost:2181
Topic:test PartitionCount:1 ReplicationFactor:1 Configs:
Topic: test Partition: 0 Leader: 9092 Replicas: 9092 Isr: 9092
➜ kafka_2.9.2-0.8.2.0
➜ kafka-docker git:(master) ✗ fig ps
Name Command State Ports
-----------------------------------------------------------------------------------------
kafkadocker_kafka_1 /bin/sh -c start- Up 0.0.0.0:9092->9092/t
kafka.sh cp
kafkadocker_zookeepe /bin/sh -c Up 0.0.0.0:2181->2181/t
r_1 /usr/sbin/sshd ... cp, 22/tcp,
2888/tcp, 3888/tcp
➜ kafka-docker git:(master) ✗ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
65fc3ba88cc5 wurstmeister/kafka:0.8.2.0 "/bin/sh -c start-ka 3 minutes ago Up 3 minutes 0.0.0.0:9092->9092/tcp kafkadocker_kafka_1
2e91593f0fa2 wurstmeister/zookeeper:latest "/bin/sh -c '/usr/sb 3 minutes ago Up 3 minutes 22/tcp, 2888/tcp, 3888/tcp, 0.0.0.0:2181->2181/tcp kafkadocker_zookeeper_1
➜ kafka-docker git:(master) ✗
Network config:
➜ kafka-docker git:(master) ✗ ifconfig
docker0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 172.17.42.1 netmask 255.255.0.0 broadcast 0.0.0.0
inet6 fe80::5484:7aff:fefe:9799 prefixlen 64 scopeid 0x20<link>
ether 56:84:7a:fe:97:99 txqueuelen 0 (Ethernet)
RX packets 823 bytes 50199 (49.0 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 796 bytes 78010 (76.1 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
enp60s0: flags=4099<UP,BROADCAST,MULTICAST> mtu 1500
ether ac:9e:17:91:2c:79 txqueuelen 1000 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536
inet 127.0.0.1 netmask 255.0.0.0
inet6 ::1 prefixlen 128 scopeid 0x10<host>
loop txqueuelen 0 (Local Loopback)
RX packets 687 bytes 62166 (60.7 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 687 bytes 62166 (60.7 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
veth9b1087e: flags=67<UP,BROADCAST,RUNNING> mtu 1500
inet6 fe80::8cdc:46ff:fefc:5815 prefixlen 64 scopeid 0x20<link>
ether 8e:dc:46:fc:58:15 txqueuelen 0 (Ethernet)
RX packets 732 bytes 56240 (54.9 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 546 bytes 53475 (52.2 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
vethe4c6150: flags=67<UP,BROADCAST,RUNNING> mtu 1500
inet6 fe80::c57:fcff:fea3:187b prefixlen 64 scopeid 0x20<link>
ether 0e:57:fc:a3:18:7b txqueuelen 0 (Ethernet)
RX packets 266 bytes 25639 (25.0 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 473 bytes 40864 (39.9 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
wls1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 192.168.1.76 netmask 255.255.255.0 broadcast 192.168.1.255
inet6 fe80::2ab2:bdff:feef:bb18 prefixlen 64 scopeid 0x20<link>
ether 28:b2:bd:ef:bb:18 txqueuelen 1000 (Ethernet)
RX packets 105087 bytes 128139068 (122.2 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 69403 bytes 12466977 (11.8 MiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
➜ kafka-docker git:(master) ✗ cat fig-single-broker.yml
zookeeper:
image: wurstmeister/zookeeper
ports:
- "2181:2181"
kafka:
image: wurstmeister/kafka:0.8.2.0
ports:
- "9092:9092"
links:
- zookeeper:zk
environment:
KAFKA_ADVERTISED_HOST_NAME: 192.168.1.76
volumes:
- /var/run/docker.sock:/var/run/docker.sock
kafka version:
➜ kafka-docker git:(master) ✗ docker version
Client version: 1.5.0
Client API version: 1.17
Go version (client): go1.3.3
Git commit (client): a8a31ef/1.5.0
OS/Arch (client): linux/amd64
Server version: 1.5.0
Server API version: 1.17
Go version (server): go1.3.3
Git commit (server): a8a31ef/1.5.0
Sorry, should have included that too. Appologies.
setup:
➜ kafka-docker git:(master) ✗ fig -f fig-single-broker.yml up -d
Creating kafkadocker_zookeeper_1...
Creating kafkadocker_kafka_1...
➜ kafka-docker git:(master) ✗ cat fig-single-broker.yml
zookeeper:
image: wurstmeister/zookeeper
ports:
- "2181:2181"
kafka:
image: wurstmeister/kafka:0.8.2.0
ports:
- "9092:9092"
links:
- zookeeper:zk
environment:
KAFKA_ADVERTISED_HOST_NAME: 192.168.1.76
volumes:
- /var/run/docker.sock:/var/run/docker.sock
Running local docker tools against what should now be running on localhost 2171 / 9092:
➜ kafka_2.9.2-0.8.2.0 bin/kafka-topics.sh --list --zookeeper 192.168.1.76:2181
➜ kafka_2.9.2-0.8.2.0 bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic test
Created topic "test".
➜ kafka_2.9.2-0.8.2.0 bin/kafka-topics.sh --list --zookeeper 192.168.1.76:2181
➜ kafka_2.9.2-0.8.2.0 bin/kafka-console-producer.sh --broker-list localhost:9092 --topic test
[2015-02-23 10:20:42,411] WARN Property topic is not valid (kafka.utils.VerifiableProperties)
[2015-02-23 10:20:44,605] WARN Error while fetching metadata [{TopicMetadata for topic test ->
No partition metadata for topic test due to kafka.common.LeaderNotAvailableException}] for topic [test]: class kafka.common.LeaderNotAvailableException (kafka.producer.BrokerPartitionInfo)
[2015-02-23 10:20:44,612] WARN Error while fetching metadata [{TopicMetadata for topic test ->
No partition metadata for topic test due to kafka.common.LeaderNotAvailableException}] for topic [test]: class kafka.common.LeaderNotAvailableException (kafka.producer.BrokerPartitionInfo)
[2015-02-23 10:20:44,613] ERROR Failed to collate messages by topic, partition due to: Failed to fetch topic metadata for topic: test (kafka.producer.async.DefaultEventHandler)
[2015-02-23 10:20:44,719] WARN Error while fetching metadata [{TopicMetadata for topic test ->
No partition metadata for topic test due to kafka.common.LeaderNotAvailableException}] for topic [test]: class kafka.common.LeaderNotAvailableException (kafka.producer.BrokerPartitionInfo)
[2015-02-23 10:20:44,723] WARN Error while fetching metadata [{TopicMetadata for topic test ->
No partition metadata for topic test due to kafka.common.LeaderNotAvailableException}] for topic [test]: class kafka.common.LeaderNotAvailableException (kafka.producer.BrokerPartitionInfo)
[2015-02-23 10:20:44,723] ERROR Failed to collate messages by topic, partition due to: Failed to fetch topic metadata for topic: test (kafka.producer.async.DefaultEventHandler)
[2015-02-23 10:20:44,828] WARN Error while fetching metadata [{TopicMetadata for topic test ->
No partition metadata for topic test due to kafka.common.LeaderNotAvailableException}] for topic [test]: class kafka.common.LeaderNotAvailableException (kafka.producer.BrokerPartitionInfo)
[2015-02-23 10:20:44,832] WARN Error while fetching metadata [{TopicMetadata for topic test ->
No partition metadata for topic test due to kafka.common.LeaderNotAvailableException}] for topic [test]: class kafka.common.LeaderNotAvailableException (kafka.producer.BrokerPartitionInfo)
[2015-02-23 10:20:44,833] ERROR Failed to collate messages by topic, partition due to: Failed to fetch topic metadata for topic: test (kafka.producer.async.DefaultEventHandler)
[2015-02-23 10:20:44,938] WARN Error while fetching metadata [{TopicMetadata for topic test ->
No partition metadata for topic test due to kafka.common.LeaderNotAvailableException}] for topic [test]: class kafka.common.LeaderNotAvailableException (kafka.producer.BrokerPartitionInfo)
[2015-02-23 10:20:44,944] WARN Error while fetching metadata [{TopicMetadata for topic test ->
No partition metadata for topic test due to kafka.common.LeaderNotAvailableException}] for topic [test]: class kafka.common.LeaderNotAvailableException (kafka.producer.BrokerPartitionInfo)
[2015-02-23 10:20:44,944] ERROR Failed to collate messages by topic, partition due to: Failed to fetch topic metadata for topic: test (kafka.producer.async.DefaultEventHandler)
[2015-02-23 10:20:45,048] WARN Error while fetching metadata [{TopicMetadata for topic test ->
No partition metadata for topic test due to kafka.common.LeaderNotAvailableException}] for topic [test]: class kafka.common.LeaderNotAvailableException (kafka.producer.BrokerPartitionInfo)
[2015-02-23 10:20:45,050] ERROR Failed to send requests for topics test with correlation ids in [0,8] (kafka.producer.async.DefaultEventHandler)
[2015-02-23 10:20:45,051] ERROR Error in handling batch of 1 events (kafka.producer.async.ProducerSendThread)
kafka.common.FailedToSendMessageException: Failed to send messages after 3 tries.
at kafka.producer.async.DefaultEventHandler.handle(DefaultEventHandler.scala:90)
at kafka.producer.async.ProducerSendThread.tryToHandle(ProducerSendThread.scala:105)
at kafka.producer.async.ProducerSendThread$$anonfun$processEvents$3.apply(ProducerSendThread.scala:88)
at kafka.producer.async.ProducerSendThread$$anonfun$processEvents$3.apply(ProducerSendThread.scala:68)
at scala.collection.immutable.Stream.foreach(Stream.scala:526)
at kafka.producer.async.ProducerSendThread.processEvents(ProducerSendThread.scala:67)
at kafka.producer.async.ProducerSendThread.run(ProducerSendThread.scala:45)
^C% ➜ kafka_2.9.2-0.8.2.0
This is my producer debug output (running sarama golang client)
➜ LegacyProducer cat sarama.debug.log
PREFIX: 2015/02/23 10:22:41 client.go:42: Initializing new client
PREFIX: 2015/02/23 10:22:41 client.go:353: Fetching metadata for [] from broker localhost:9092
PREFIX: 2015/02/23 10:22:41 broker.go:112: Connected to broker localhost:9092
PREFIX: 2015/02/23 10:22:41 client.go:82: Successfully initialized new client
PREFIX: 2015/02/23 10:22:41 client.go:353: Fetching metadata for [my_new_topic] from broker localhost:9092
PREFIX: 2015/02/23 10:22:41 client.go:366: Some partitions are leaderless, waiting 250ms for election... (3 retries remaining)
PREFIX: 2015/02/23 10:22:41 client.go:353: Fetching metadata for [my_new_topic] from broker localhost:9092
PREFIX: 2015/02/23 10:22:41 client.go:366: Some partitions are leaderless, waiting 250ms for election... (2 retries remaining)
PREFIX: 2015/02/23 10:22:42 client.go:353: Fetching metadata for [my_new_topic] from broker localhost:9092
PREFIX: 2015/02/23 10:22:42 client.go:366: Some partitions are leaderless, waiting 250ms for election... (1 retries remaining)
PREFIX: 2015/02/23 10:22:42 client.go:353: Fetching metadata for [my_new_topic] from broker localhost:9092
PREFIX: 2015/02/23 10:22:42 client.go:363: Some partitions are leaderless, but we're out of retries
PREFIX: 2015/02/23 10:22:42 producer.go:280: Producer shutting down.
PREFIX: 2015/02/23 10:22:42 client.go:101: Closing Client
fig logs:
Attaching to kafkadocker_kafka_1, kafkadocker_zookeeper_1
�[36mkafka_1 | �[0m[2015-02-23 18:20:03,473] INFO Verifying properties (kafka.utils.VerifiableProperties)
�[36mkafka_1 | �[0m[2015-02-23 18:20:03,498] INFO Property advertised.host.name is overridden to 192.168.1.76 (kafka.utils.VerifiableProperties)
�[36mkafka_1 | �[0m[2015-02-23 18:20:03,499] INFO Property advertised.port is overridden to 9092 (kafka.utils.VerifiableProperties)
�[36mkafka_1 | �[0m[2015-02-23 18:20:03,499] INFO Property broker.id is overridden to 9092 (kafka.utils.VerifiableProperties)
�[36mkafka_1 | �[0m[2015-02-23 18:20:03,499] INFO Property log.cleaner.enable is overridden to false (kafka.utils.VerifiableProperties)
�[36mkafka_1 | �[0m[2015-02-23 18:20:03,499] INFO Property log.dirs is overridden to /kafka/kafka-logs-9092 (kafka.utils.VerifiableProperties)
�[36mkafka_1 | �[0m[2015-02-23 18:20:03,499] INFO Property log.retention.check.interval.ms is overridden to 300000 (kafka.utils.VerifiableProperties)
�[36mkafka_1 | �[0m[2015-02-23 18:20:03,499] INFO Property log.retention.hours is overridden to 168 (kafka.utils.VerifiableProperties)
�[36mkafka_1 | �[0m[2015-02-23 18:20:03,500] INFO Property log.segment.bytes is overridden to 1073741824 (kafka.utils.VerifiableProperties)
�[36mkafka_1 | �[0m[2015-02-23 18:20:03,500] INFO Property num.io.threads is overridden to 8 (kafka.utils.VerifiableProperties)
�[36mkafka_1 | �[0m[2015-02-23 18:20:03,500] INFO Property num.network.threads is overridden to 3 (kafka.utils.VerifiableProperties)
�[36mkafka_1 | �[0m[2015-02-23 18:20:03,500] INFO Property num.partitions is overridden to 1 (kafka.utils.VerifiableProperties)
�[36mkafka_1 | �[0m[2015-02-23 18:20:03,500] INFO Property num.recovery.threads.per.data.dir is overridden to 1 (kafka.utils.VerifiableProperties)
�[36mkafka_1 | �[0m[2015-02-23 18:20:03,500] INFO Property port is overridden to 9092 (kafka.utils.VerifiableProperties)
�[36mkafka_1 | �[0m[2015-02-23 18:20:03,500] INFO Property socket.receive.buffer.bytes is overridden to 102400 (kafka.utils.VerifiableProperties)
�[36mkafka_1 | �[0m[2015-02-23 18:20:03,501] INFO Property socket.request.max.bytes is overridden to 104857600 (kafka.utils.VerifiableProperties)
�[36mkafka_1 | �[0m[2015-02-23 18:20:03,501] INFO Property socket.send.buffer.bytes is overridden to 102400 (kafka.utils.VerifiableProperties)
�[36mkafka_1 | �[0m[2015-02-23 18:20:03,501] INFO Property zookeeper.connect is overridden to 172.17.0.2:2181 (kafka.utils.VerifiableProperties)
�[36mkafka_1 | �[0m[2015-02-23 18:20:03,501] INFO Property zookeeper.connection.timeout.ms is overridden to 6000 (kafka.utils.VerifiableProperties)
�[36mkafka_1 | �[0m[2015-02-23 18:20:03,530] INFO [Kafka Server 9092], starting (kafka.server.KafkaServer)
�[36mkafka_1 | �[0m[2015-02-23 18:20:03,532] INFO [Kafka Server 9092], Connecting to zookeeper on 172.17.0.2:2181 (kafka.server.KafkaServer)
�[36mkafka_1 | �[0m[2015-02-23 18:20:03,538] INFO Starting ZkClient event thread. (org.I0Itec.zkclient.ZkEventThread)
�[36mkafka_1 | �[0m[2015-02-23 18:20:03,543] INFO Client environment:zookeeper.version=3.4.6-1569965, built on 02/20/2014 09:09 GMT (org.apache.zookeeper.ZooKeeper)
�[36mkafka_1 | �[0m[2015-02-23 18:20:03,543] INFO Client environment:host.name=4b7e696a10e0 (org.apache.zookeeper.ZooKeeper)
�[36mkafka_1 | �[0m[2015-02-23 18:20:03,543] INFO Client environment:java.version=1.6.0_34 (org.apache.zookeeper.ZooKeeper)
�[36mkafka_1 | �[0m[2015-02-23 18:20:03,543] INFO Client environment:java.vendor=Sun Microsystems Inc. (org.apache.zookeeper.ZooKeeper)
�[36mkafka_1 | �[0m[2015-02-23 18:20:03,543] INFO Client environment:java.home=/usr/lib/jvm/java-6-openjdk-amd64/jre (org.apache.zookeeper.ZooKeeper)
�[36mkafka_1 | �[0m[2015-02-23 18:20:03,543] INFO Client environment:java.class.path=:/opt/kafka_2.10-0.8.2.0/bin/../core/build/dependant-libs-2.10.4*/*.jar:/opt/kafka_2.10-0.8.2.0/bin/../examples/build/libs//kafka-examples*.jar:/opt/kafka_2.10-0.8.2.0/bin/../contrib/hadoop-consumer/build/libs//kafka-hadoop-consumer*.jar:/opt/kafka_2.10-0.8.2.0/bin/../contrib/hadoop-producer/build/libs//kafka-hadoop-producer*.jar:/opt/kafka_2.10-0.8.2.0/bin/../clients/build/libs/kafka-clients*.jar:/opt/kafka_2.10-0.8.2.0/bin/../libs/jopt-simple-3.2.jar:/opt/kafka_2.10-0.8.2.0/bin/../libs/kafka-clients-0.8.2.0.jar:/opt/kafka_2.10-0.8.2.0/bin/../libs/kafka_2.10-0.8.2.0-javadoc.jar:/opt/kafka_2.10-0.8.2.0/bin/../libs/kafka_2.10-0.8.2.0-scaladoc.jar:/opt/kafka_2.10-0.8.2.0/bin/../libs/kafka_2.10-0.8.2.0-sources.jar:/opt/kafka_2.10-0.8.2.0/bin/../libs/kafka_2.10-0.8.2.0-test.jar:/opt/kafka_2.10-0.8.2.0/bin/../libs/kafka_2.10-0.8.2.0.jar:/opt/kafka_2.10-0.8.2.0/bin/../libs/log4j-1.2.16.jar:/opt/kafka_2.10-0.8.2.0/bin/../libs/lz4-1.2.0.jar:/opt/kafka_2.10-0.8.2.0/bin/../libs/metrics-core-2.2.0.jar:/opt/kafka_2.10-0.8.2.0/bin/../libs/scala-library-2.10.4.jar:/opt/kafka_2.10-0.8.2.0/bin/../libs/slf4j-api-1.7.6.jar:/opt/kafka_2.10-0.8.2.0/bin/../libs/slf4j-log4j12-1.6.1.jar:/opt/kafka_2.10-0.8.2.0/bin/../libs/snappy-java-1.1.1.6.jar:/opt/kafka_2.10-0.8.2.0/bin/../libs/zkclient-0.3.jar:/opt/kafka_2.10-0.8.2.0/bin/../libs/zookeeper-3.4.6.jar:/opt/kafka_2.10-0.8.2.0/bin/../core/build/libs/kafka_2.10*.jar (org.apache.zookeeper.ZooKeeper)
�[36mkafka_1 | �[0m[2015-02-23 18:20:03,543] INFO Client environment:java.library.path=/usr/lib/jvm/java-6-openjdk-amd64/jre/lib/amd64/server:/usr/lib/jvm/java-6-openjdk-amd64/jre/lib/amd64:/usr/lib/jvm/java-6-openjdk-amd64/jre/../lib/amd64:/usr/java/packages/lib/amd64:/usr/lib/x86_64-linux-gnu/jni:/lib/x86_64-linux-gnu:/usr/lib/x86_64-linux-gnu:/usr/lib/jni:/lib:/usr/lib (org.apache.zookeeper.ZooKeeper)
�[36mkafka_1 | �[0m[2015-02-23 18:20:03,543] INFO Client environment:java.io.tmpdir=/tmp (org.apache.zookeeper.ZooKeeper)
�[36mkafka_1 | �[0m[2015-02-23 18:20:03,543] INFO Client environment:java.compiler=<NA> (org.apache.zookeeper.ZooKeeper)
�[36mkafka_1 | �[0m[2015-02-23 18:20:03,543] INFO Client environment:os.name=Linux (org.apache.zookeeper.ZooKeeper)
�[36mkafka_1 | �[0m[2015-02-23 18:20:03,543] INFO Client environment:os.arch=amd64 (org.apache.zookeeper.ZooKeeper)
�[36mkafka_1 | �[0m[2015-02-23 18:20:03,543] INFO Client environment:os.version=3.18.7-200.fc21.x86_64 (org.apache.zookeeper.ZooKeeper)
�[36mkafka_1 | �[0m[2015-02-23 18:20:03,543] INFO Client environment:user.name=root (org.apache.zookeeper.ZooKeeper)
�[36mkafka_1 | �[0m[2015-02-23 18:20:03,543] INFO Client environment:user.home=/root (org.apache.zookeeper.ZooKeeper)
�[36mkafka_1 | �[0m[2015-02-23 18:20:03,543] INFO Client environment:user.dir=/ (org.apache.zookeeper.ZooKeeper)
�[36mkafka_1 | �[0m[2015-02-23 18:20:03,544] INFO Initiating client connection, connectString=172.17.0.2:2181 sessionTimeout=6000 watcher=org.I0Itec.zkclient.ZkClient@1cf1df22 (org.apache.zookeeper.ZooKeeper)
�[36mkafka_1 | �[0m[2015-02-23 18:20:03,567] INFO Opening socket connection to server 172.17.0.2/172.17.0.2:2181. Will not attempt to authenticate using SASL (unknown error) (org.apache.zookeeper.ClientCnxn)
�[36mkafka_1 | �[0m[2015-02-23 18:20:03,572] INFO Socket connection established to 172.17.0.2/172.17.0.2:2181, initiating session (org.apache.zookeeper.ClientCnxn)
�[36mkafka_1 | �[0m[2015-02-23 18:20:03,610] INFO Session establishment complete on server 172.17.0.2/172.17.0.2:2181, sessionid = 0x14bb7aadbe90000, negotiated timeout = 6000 (org.apache.zookeeper.ClientCnxn)
�[36mkafka_1 | �[0m[2015-02-23 18:20:03,612] INFO zookeeper state changed (SyncConnected) (org.I0Itec.zkclient.ZkClient)
�[36mkafka_1 | �[0m[2015-02-23 18:20:03,807] INFO Log directory '/kafka/kafka-logs-9092' not found, creating it. (kafka.log.LogManager)
�[36mkafka_1 | �[0m[2015-02-23 18:20:03,814] INFO Loading logs. (kafka.log.LogManager)
�[36mkafka_1 | �[0m[2015-02-23 18:20:03,819] INFO Logs loading complete. (kafka.log.LogManager)
�[36mkafka_1 | �[0m[2015-02-23 18:20:03,819] INFO Starting log cleanup with a period of 300000 ms. (kafka.log.LogManager)
�[36mkafka_1 | �[0m[2015-02-23 18:20:03,821] INFO Starting log flusher with a default period of 9223372036854775807 ms. (kafka.log.LogManager)
�[36mkafka_1 | �[0m[2015-02-23 18:20:03,843] INFO Awaiting socket connections on 0.0.0.0:9092. (kafka.network.Acceptor)
�[36mkafka_1 | �[0m[2015-02-23 18:20:03,844] INFO [Socket Server on Broker 9092], Started (kafka.network.SocketServer)
�[36mkafka_1 | �[0m[2015-02-23 18:20:03,895] INFO Will not load MX4J, mx4j-tools.jar is not in the classpath (kafka.utils.Mx4jLoader$)
�[36mkafka_1 | �[0m[2015-02-23 18:20:03,930] INFO 9092 successfully elected as leader (kafka.server.ZookeeperLeaderElector)
�[36mkafka_1 | �[0m[2015-02-23 18:20:04,020] INFO Registered broker 9092 at path /brokers/ids/9092 with address 192.168.1.76:9092. (kafka.utils.ZkUtils$)
�[36mkafka_1 | �[0m[2015-02-23 18:20:04,033] INFO [Kafka Server 9092], started (kafka.server.KafkaServer)
�[36mkafka_1 | �[0m[2015-02-23 18:20:04,076] INFO New leader is 9092 (kafka.server.ZookeeperLeaderElector$LeaderChangeListener)
�[36mkafka_1 | �[0m[2015-02-23 18:20:44,617] INFO Closing socket connection to /172.17.42.1. (kafka.network.Processor)
�[36mkafka_1 | �[0m[2015-02-23 18:20:44,617] INFO Closing socket connection to /172.17.42.1. (kafka.network.Processor)
�[36mkafka_1 | �[0m[2015-02-23 18:20:44,719] INFO Closing socket connection to /172.17.42.1. (kafka.network.Processor)
�[36mkafka_1 | �[0m[2015-02-23 18:20:44,723] INFO Closing socket connection to /172.17.42.1. (kafka.network.Processor)
�[36mkafka_1 | �[0m[2015-02-23 18:20:44,828] INFO Closing socket connection to /172.17.42.1. (kafka.network.Processor)
�[36mkafka_1 | �[0m[2015-02-23 18:20:44,832] INFO Closing socket connection to /172.17.42.1. (kafka.network.Processor)
�[36mkafka_1 | �[0m[2015-02-23 18:20:44,937] INFO Closing socket connection to /172.17.42.1. (kafka.network.Processor)
�[36mkafka_1 | �[0m[2015-02-23 18:20:44,943] INFO Closing socket connection to /172.17.42.1. (kafka.network.Processor)
�[36mkafka_1 | �[0m[2015-02-23 18:20:45,048] INFO Closing socket connection to /172.17.42.1. (kafka.network.Processor)
�[36mkafka_1 | �[0m[2015-02-23 18:22:41,521] INFO Topic creation {"version":1,"partitions":{"0":[9092]}} (kafka.admin.AdminUtils$)
�[36mkafka_1 | �[0m[2015-02-23 18:22:41,534] INFO [KafkaApi-9092] Auto creation of topic my_new_topic with 1 partitions and replication factor 1 is successful! (kafka.server.KafkaApis)
�[36mkafka_1 | �[0m[2015-02-23 18:22:42,296] INFO Closing socket connection to /172.17.42.1. (kafka.network.Processor)
�[33mzookeeper_1 | �[0mJMX enabled by default
�[33mzookeeper_1 | �[0mUsing config: /opt/zookeeper-3.4.6/bin/../conf/zoo.cfg
�[33mzookeeper_1 | �[0m2015-02-23 18:20:02,853 [myid:] - INFO [main:QuorumPeerConfig@103] - Reading configuration from: /opt/zookeeper-3.4.6/bin/../conf/zoo.cfg
�[33mzookeeper_1 | �[0m2015-02-23 18:20:02,858 [myid:] - INFO [main:DatadirCleanupManager@78] - autopurge.snapRetainCount set to 3
�[33mzookeeper_1 | �[0m2015-02-23 18:20:02,859 [myid:] - INFO [main:DatadirCleanupManager@79] - autopurge.purgeInterval set to 1
�[33mzookeeper_1 | �[0m2015-02-23 18:20:02,859 [myid:] - WARN [main:QuorumPeerMain@113] - Either no config or no quorum defined in config, running in standalone mode
�[33mzookeeper_1 | �[0m2015-02-23 18:20:02,859 [myid:] - INFO [PurgeTask:DatadirCleanupManager$PurgeTask@138] - Purge task started.
�[33mzookeeper_1 | �[0m2015-02-23 18:20:02,869 [myid:] - INFO [PurgeTask:DatadirCleanupManager$PurgeTask@144] - Purge task completed.
�[33mzookeeper_1 | �[0m2015-02-23 18:20:02,875 [myid:] - INFO [main:QuorumPeerConfig@103] - Reading configuration from: /opt/zookeeper-3.4.6/bin/../conf/zoo.cfg
�[33mzookeeper_1 | �[0m2015-02-23 18:20:02,875 [myid:] - INFO [main:ZooKeeperServerMain@95] - Starting server
�[33mzookeeper_1 | �[0m2015-02-23 18:20:02,883 [myid:] - INFO [main:Environment@100] - Server environment:zookeeper.version=3.4.6-1569965, built on 02/20/2014 09:09 GMT
�[33mzookeeper_1 | �[0m2015-02-23 18:20:02,883 [myid:] - INFO [main:Environment@100] - Server environment:host.name=a97404af716f
�[33mzookeeper_1 | �[0m2015-02-23 18:20:02,884 [myid:] - INFO [main:Environment@100] - Server environment:java.version=1.7.0_65
�[33mzookeeper_1 | �[0m2015-02-23 18:20:02,884 [myid:] - INFO [main:Environment@100] - Server environment:java.vendor=Oracle Corporation
�[33mzookeeper_1 | �[0m2015-02-23 18:20:02,884 [myid:] - INFO [main:Environment@100] - Server environment:java.home=/usr/lib/jvm/java-7-openjdk-amd64/jre
�[33mzookeeper_1 | �[0m2015-02-23 18:20:02,884 [myid:] - INFO [main:Environment@100] - Server environment:java.class.path=/opt/zookeeper-3.4.6/bin/../build/classes:/opt/zookeeper-3.4.6/bin/../build/lib/*.jar:/opt/zookeeper-3.4.6/bin/../lib/slf4j-log4j12-1.6.1.jar:/opt/zookeeper-3.4.6/bin/../lib/slf4j-api-1.6.1.jar:/opt/zookeeper-3.4.6/bin/../lib/netty-3.7.0.Final.jar:/opt/zookeeper-3.4.6/bin/../lib/log4j-1.2.16.jar:/opt/zookeeper-3.4.6/bin/../lib/jline-0.9.94.jar:/opt/zookeeper-3.4.6/bin/../zookeeper-3.4.6.jar:/opt/zookeeper-3.4.6/bin/../src/java/lib/*.jar:/opt/zookeeper-3.4.6/bin/../conf:
�[33mzookeeper_1 | �[0m2015-02-23 18:20:02,884 [myid:] - INFO [main:Environment@100] - Server environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib/x86_64-linux-gnu/jni:/lib/x86_64-linux-gnu:/usr/lib/x86_64-linux-gnu:/usr/lib/jni:/lib:/usr/lib
�[33mzookeeper_1 | �[0m2015-02-23 18:20:02,884 [myid:] - INFO [main:Environment@100] - Server environment:java.io.tmpdir=/tmp
�[33mzookeeper_1 | �[0m2015-02-23 18:20:02,886 [myid:] - INFO [main:Environment@100] - Server environment:java.compiler=<NA>
�[33mzookeeper_1 | �[0m2015-02-23 18:20:02,886 [myid:] - INFO [main:Environment@100] - Server environment:os.name=Linux
�[33mzookeeper_1 | �[0m2015-02-23 18:20:02,886 [myid:] - INFO [main:Environment@100] - Server environment:os.arch=amd64
�[33mzookeeper_1 | �[0m2015-02-23 18:20:02,887 [myid:] - INFO [main:Environment@100] - Server environment:os.version=3.18.7-200.fc21.x86_64
�[33mzookeeper_1 | �[0m2015-02-23 18:20:02,887 [myid:] - INFO [main:Environment@100] - Server environment:user.name=root
�[33mzookeeper_1 | �[0m2015-02-23 18:20:02,887 [myid:] - INFO [main:Environment@100] - Server environment:user.home=/root
�[33mzookeeper_1 | �[0m2015-02-23 18:20:02,887 [myid:] - INFO [main:Environment@100] - Server environment:user.dir=/opt/zookeeper-3.4.6
�[33mzookeeper_1 | �[0m2015-02-23 18:20:02,888 [myid:] - INFO [main:ZooKeeperServer@755] - tickTime set to 2000
�[33mzookeeper_1 | �[0m2015-02-23 18:20:02,888 [myid:] - INFO [main:ZooKeeperServer@764] - minSessionTimeout set to -1
�[33mzookeeper_1 | �[0m2015-02-23 18:20:02,888 [myid:] - INFO [main:ZooKeeperServer@773] - maxSessionTimeout set to -1
�[33mzookeeper_1 | �[0m2015-02-23 18:20:02,906 [myid:] - INFO [main:NIOServerCnxnFactory@94] - binding to port 0.0.0.0/0.0.0.0:2181
�[33mzookeeper_1 | �[0m2015-02-23 18:20:03,573 [myid:] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxnFactory@197] - Accepted socket connection from /172.17.0.3:39117
�[33mzookeeper_1 | �[0m2015-02-23 18:20:03,578 [myid:] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:ZooKeeperServer@868] - Client attempting to establish new session at /172.17.0.3:39117
�[33mzookeeper_1 | �[0m2015-02-23 18:20:03,580 [myid:] - INFO [SyncThread:0:FileTxnLog@199] - Creating new log file: log.1
�[33mzookeeper_1 | �[0m2015-02-23 18:20:03,609 [myid:] - INFO [SyncThread:0:ZooKeeperServer@617] - Established session 0x14bb7aadbe90000 with negotiated timeout 6000 for client /172.17.0.3:39117
�[33mzookeeper_1 | �[0m2015-02-23 18:20:03,650 [myid:] - INFO [ProcessThread(sid:0 cport:-1)::PrepRequestProcessor@645] - Got user-level KeeperException when processing sessionid:0x14bb7aadbe90000 type:create cxid:0x4 zxid:0x3 txntype:-1 reqpath:n/a Error Path:/brokers Error:KeeperErrorCode = NoNode for /brokers
�[33mzookeeper_1 | �[0m2015-02-23 18:20:03,702 [myid:] - INFO [ProcessThread(sid:0 cport:-1)::PrepRequestProcessor@645] - Got user-level KeeperException when processing sessionid:0x14bb7aadbe90000 type:create cxid:0xa zxid:0x7 txntype:-1 reqpath:n/a Error Path:/config Error:KeeperErrorCode = NoNode for /config
�[33mzookeeper_1 | �[0m2015-02-23 18:20:03,746 [myid:] - INFO [ProcessThread(sid:0 cport:-1)::PrepRequestProcessor@645] - Got user-level KeeperException when processing sessionid:0x14bb7aadbe90000 type:create cxid:0x10 zxid:0xb txntype:-1 reqpath:n/a Error Path:/admin Error:KeeperErrorCode = NoNode for /admin
�[33mzookeeper_1 | �[0m2015-02-23 18:20:03,933 [myid:] - INFO [ProcessThread(sid:0 cport:-1)::PrepRequestProcessor@645] - Got user-level KeeperException when processing sessionid:0x14bb7aadbe90000 type:setData cxid:0x1a zxid:0xf txntype:-1 reqpath:n/a Error Path:/controller_epoch Error:KeeperErrorCode = NoNode for /controller_epoch
�[33mzookeeper_1 | �[0m2015-02-23 18:20:03,986 [myid:] - INFO [ProcessThread(sid:0 cport:-1)::PrepRequestProcessor@645] - Got user-level KeeperException when processing sessionid:0x14bb7aadbe90000 type:delete cxid:0x27 zxid:0x11 txntype:-1 reqpath:n/a Error Path:/admin/preferred_replica_election Error:KeeperErrorCode = NoNode for /admin/preferred_replica_election
�[33mzookeeper_1 | �[0m2015-02-23 18:20:24,441 [myid:] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxnFactory@197] - Accepted socket connection from /192.168.1.76:45181
�[33mzookeeper_1 | �[0m2015-02-23 18:20:24,443 [myid:] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:ZooKeeperServer@868] - Client attempting to establish new session at /192.168.1.76:45181
�[33mzookeeper_1 | �[0m2015-02-23 18:20:24,453 [myid:] - INFO [SyncThread:0:ZooKeeperServer@617] - Established session 0x14bb7aadbe90001 with negotiated timeout 30000 for client /192.168.1.76:45181
�[33mzookeeper_1 | �[0m2015-02-23 18:20:24,484 [myid:] - INFO [ProcessThread(sid:0 cport:-1)::PrepRequestProcessor@494] - Processed session termination for sessionid: 0x14bb7aadbe90001
�[33mzookeeper_1 | �[0m2015-02-23 18:20:24,493 [myid:] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn@1007] - Closed socket connection for client /192.168.1.76:45181 which had sessionid 0x14bb7aadbe90001
�[33mzookeeper_1 | �[0m2015-02-23 18:20:32,555 [myid:] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxnFactory@197] - Accepted socket connection from /172.17.42.1:50364
�[33mzookeeper_1 | �[0m2015-02-23 18:20:32,558 [myid:] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:ZooKeeperServer@868] - Client attempting to establish new session at /172.17.42.1:50364
�[33mzookeeper_1 | �[0m2015-02-23 18:20:32,569 [myid:] - INFO [SyncThread:0:ZooKeeperServer@617] - Established session 0x14bb7aadbe90002 with negotiated timeout 30000 for client /172.17.42.1:50364
�[33mzookeeper_1 | �[0m2015-02-23 18:20:32,702 [myid:] - INFO [ProcessThread(sid:0 cport:-1)::PrepRequestProcessor@645] - Got user-level KeeperException when processing sessionid:0x14bb7aadbe90002 type:setData cxid:0x3 zxid:0x16 txntype:-1 reqpath:n/a Error Path:/config/topics/test Error:KeeperErrorCode = NoNode for /config/topics/test
�[33mzookeeper_1 | �[0m2015-02-23 18:20:32,720 [myid:] - INFO [ProcessThread(sid:0 cport:-1)::PrepRequestProcessor@645] - Got user-level KeeperException when processing sessionid:0x14bb7aadbe90002 type:create cxid:0x4 zxid:0x17 txntype:-1 reqpath:n/a Error Path:/config/topics Error:KeeperErrorCode = NodeExists for /config/topics
�[33mzookeeper_1 | �[0m2015-02-23 18:20:32,775 [myid:] - INFO [ProcessThread(sid:0 cport:-1)::PrepRequestProcessor@494] - Processed session termination for sessionid: 0x14bb7aadbe90002
�[33mzookeeper_1 | �[0m2015-02-23 18:20:32,784 [myid:] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn@1007] - Closed socket connection for client /172.17.42.1:50364 which had sessionid 0x14bb7aadbe90002
�[33mzookeeper_1 | �[0m2015-02-23 18:20:32,807 [myid:] - INFO [ProcessThread(sid:0 cport:-1)::PrepRequestProcessor@645] - Got user-level KeeperException when processing sessionid:0x14bb7aadbe90000 type:create cxid:0x37 zxid:0x1b txntype:-1 reqpath:n/a Error Path:/brokers/topics/test/partitions/0 Error:KeeperErrorCode = NoNode for /brokers/topics/test/partitions/0
�[33mzookeeper_1 | �[0m2015-02-23 18:20:32,818 [myid:] - INFO [ProcessThread(sid:0 cport:-1)::PrepRequestProcessor@645] - Got user-level KeeperException when processing sessionid:0x14bb7aadbe90000 type:create cxid:0x38 zxid:0x1c txntype:-1 reqpath:n/a Error Path:/brokers/topics/test/partitions Error:KeeperErrorCode = NoNode for /brokers/topics/test/partitions
�[33mzookeeper_1 | �[0m2015-02-23 18:20:37,280 [myid:] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxnFactory@197] - Accepted socket connection from /192.168.1.76:45188
�[33mzookeeper_1 | �[0m2015-02-23 18:20:37,283 [myid:] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:ZooKeeperServer@868] - Client attempting to establish new session at /192.168.1.76:45188
�[33mzookeeper_1 | �[0m2015-02-23 18:20:37,292 [myid:] - INFO [SyncThread:0:ZooKeeperServer@617] - Established session 0x14bb7aadbe90003 with negotiated timeout 30000 for client /192.168.1.76:45188
�[33mzookeeper_1 | �[0m2015-02-23 18:20:37,326 [myid:] - INFO [ProcessThread(sid:0 cport:-1)::PrepRequestProcessor@494] - Processed session termination for sessionid: 0x14bb7aadbe90003
�[33mzookeeper_1 | �[0m2015-02-23 18:20:37,335 [myid:] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn@1007] - Closed socket connection for client /192.168.1.76:45188 which had sessionid 0x14bb7aadbe90003
�[33mzookeeper_1 | �[0m2015-02-23 18:22:41,486 [myid:] - INFO [ProcessThread(sid:0 cport:-1)::PrepRequestProcessor@645] - Got user-level KeeperException when processing sessionid:0x14bb7aadbe90000 type:setData cxid:0x50 zxid:0x22 txntype:-1 reqpath:n/a Error Path:/config/topics/my_new_topic Error:KeeperErrorCode = NoNode for /config/topics/my_new_topic
�[33mzookeeper_1 | �[0m2015-02-23 18:22:41,497 [myid:] - INFO [ProcessThread(sid:0 cport:-1)::PrepRequestProcessor@645] - Got user-level KeeperException when processing sessionid:0x14bb7aadbe90000 type:create cxid:0x51 zxid:0x23 txntype:-1 reqpath:n/a Error Path:/config/topics Error:KeeperErrorCode = NodeExists for /config/topics
�[33mzookeeper_1 | �[0m2015-02-23 18:22:41,553 [myid:] - INFO [ProcessThread(sid:0 cport:-1)::PrepRequestProcessor@645] - Got user-level KeeperException when processing sessionid:0x14bb7aadbe90000 type:create cxid:0x5a zxid:0x26 txntype:-1 reqpath:n/a Error Path:/brokers/topics/my_new_topic/partitions/0 Error:KeeperErrorCode = NoNode for /brokers/topics/my_new_topic/partitions/0
�[33mzookeeper_1 | �[0m2015-02-23 18:22:41,564 [myid:] - INFO [ProcessThread(sid:0 cport:-1)::PrepRequestProcessor@645] - Got user-level KeeperException when processing sessionid:0x14bb7aadbe90000 type:create cxid:0x5b zxid:0x27 txntype:-1 reqpath:n/a Error Path:/brokers/topics/my_new_topic/partitions Error:KeeperErrorCode = NoNode for /brokers/topics/my_new_topic/partitions
could you please double check if the single broker setup works with the start-kafka-shell.sh script:
start-kafka-shell.sh 192.168.1.76 192.168.1.76:2181
$KAFKA_HOME/bin/kafka-console-producer.sh --broker-list 192.168.1.76:9092 --topic test
It would also be helpful to double check the state in zookeeper:
$KAFKA_HOME/bin/zookeeper-shell.sh 192.168.1.76:2181
get /brokers/topics/test/partitions/0/state
get /brokers/ids/9092
Thanks
output from kafka-console-producer.sh:
➜ kafka-docker git:(master) ✗ sudo service docker start
Redirecting to /bin/systemctl start docker.service
➜ kafka-docker git:(master) ✗ fig ps
Name Command State Ports
---------------------------------------------------------------------------
kafkadocker_kafka_1 /bin/sh -c start-kafka.sh Exit 137
kafkadocker_zookeeper_1 /bin/sh -c /usr/sbin/sshd ... Exit 137
➜ kafka-docker git:(master) ✗ fig rm
Going to remove kafkadocker_kafka_1, kafkadocker_zookeeper_1
Are you sure? [yN] y
Removing kafkadocker_zookeeper_1...
Removing kafkadocker_kafka_1...
➜ kafka-docker git:(master) ✗ fig -f fig-single-broker.yml up -d
Creating kafkadocker_zookeeper_1...
Creating kafkadocker_kafka_1...
➜ kafka-docker git:(master) ✗ cat fig-single-broker.yml
zookeeper:
image: wurstmeister/zookeeper
ports:
- "2181:2181"
kafka:
image: wurstmeister/kafka:0.8.2.0
ports:
- "9092:9092"
links:
- zookeeper:zk
environment:
KAFKA_ADVERTISED_HOST_NAME: 192.168.1.76
volumes:
- /var/run/docker.sock:/var/run/docker.sock
➜ kafka-docker git:(master) ✗ ls
broker-list.sh fig.logs fig.yml README.md start-kafka-shell.sh
Dockerfile fig-single-broker.yml LICENSE start-kafka.sh
➜ kafka-docker git:(master) ✗ ./start-kafka-shell.sh 192.168.1.76 192.168.1.76:2181
root@419a95c35482:/# $KAFKA_HOME/bin/kafka-console-producer.sh --broker-list 192.168.1.76:9092 -topic test
[2015-02-25 05:27:37,389] WARN Property topic is not valid (kafka.utils.VerifiableProperties)
[2015-02-25 05:27:52,482] WARN Fetching topic metadata with correlation id 0 for topics [Set(test)] from broker [id:0,host:192.168.1.76,port:9092] failed (kafka.client.ClientUtils$)
java.nio.channels.ClosedChannelException
at kafka.network.BlockingChannel.send(BlockingChannel.scala:100)
at kafka.producer.SyncProducer.liftedTree1$1(SyncProducer.scala:73)
at kafka.producer.SyncProducer.kafka$producer$SyncProducer$$doSend(SyncProducer.scala:72)
at kafka.producer.SyncProducer.send(SyncProducer.scala:113)
at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:58)
at kafka.producer.BrokerPartitionInfo.updateInfo(BrokerPartitionInfo.scala:82)
at kafka.producer.async.DefaultEventHandler$$anonfun$handle$1.apply$mcV$sp(DefaultEventHandler.scala:67)
at kafka.utils.Utils$.swallow(Utils.scala:172)
at kafka.utils.Logging$class.swallowError(Logging.scala:106)
at kafka.utils.Utils$.swallowError(Utils.scala:45)
at kafka.producer.async.DefaultEventHandler.handle(DefaultEventHandler.scala:67)
at kafka.producer.async.ProducerSendThread.tryToHandle(ProducerSendThread.scala:105)
at kafka.producer.async.ProducerSendThread$$anonfun$processEvents$3.apply(ProducerSendThread.scala:88)
at kafka.producer.async.ProducerSendThread$$anonfun$processEvents$3.apply(ProducerSendThread.scala:68)
at scala.collection.immutable.Stream.foreach(Stream.scala:547)
at kafka.producer.async.ProducerSendThread.processEvents(ProducerSendThread.scala:67)
at kafka.producer.async.ProducerSendThread.run(ProducerSendThread.scala:45)
[2015-02-25 05:27:52,485] ERROR fetching topic metadata for topics [Set(test)] from broker [ArrayBuffer(id:0,host:192.168.1.76,port:9092)] failed (kafka.utils.Utils$)
kafka.common.KafkaException: fetching topic metadata for topics [Set(test)] from broker [ArrayBuffer(id:0,host:192.168.1.76,port:9092)] failed
at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:72)
at kafka.producer.BrokerPartitionInfo.updateInfo(BrokerPartitionInfo.scala:82)
at kafka.producer.async.DefaultEventHandler$$anonfun$handle$1.apply$mcV$sp(DefaultEventHandler.scala:67)
at kafka.utils.Utils$.swallow(Utils.scala:172)
at kafka.utils.Logging$class.swallowError(Logging.scala:106)
at kafka.utils.Utils$.swallowError(Utils.scala:45)
at kafka.producer.async.DefaultEventHandler.handle(DefaultEventHandler.scala:67)
at kafka.producer.async.ProducerSendThread.tryToHandle(ProducerSendThread.scala:105)
at kafka.producer.async.ProducerSendThread$$anonfun$processEvents$3.apply(ProducerSendThread.scala:88)
at kafka.producer.async.ProducerSendThread$$anonfun$processEvents$3.apply(ProducerSendThread.scala:68)
at scala.collection.immutable.Stream.foreach(Stream.scala:547)
at kafka.producer.async.ProducerSendThread.processEvents(ProducerSendThread.scala:67)
at kafka.producer.async.ProducerSendThread.run(ProducerSendThread.scala:45)
Caused by: java.nio.channels.ClosedChannelException
at kafka.network.BlockingChannel.send(BlockingChannel.scala:100)
at kafka.producer.SyncProducer.liftedTree1$1(SyncProducer.scala:73)
at kafka.producer.SyncProducer.kafka$producer$SyncProducer$$doSend(SyncProducer.scala:72)
at kafka.producer.SyncProducer.send(SyncProducer.scala:113)
at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:58)
... 12 more
[2015-02-25 05:27:53,510] WARN Fetching topic metadata with correlation id 1 for topics [Set(test)] from broker [id:0,host:192.168.1.76,port:9092] failed (kafka.client.ClientUtils$)
java.nio.channels.ClosedChannelException
at kafka.network.BlockingChannel.send(BlockingChannel.scala:100)
at kafka.producer.SyncProducer.liftedTree1$1(SyncProducer.scala:73)
at kafka.producer.SyncProducer.kafka$producer$SyncProducer$$doSend(SyncProducer.scala:72)
at kafka.producer.SyncProducer.send(SyncProducer.scala:113)
at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:58)
at kafka.producer.BrokerPartitionInfo.updateInfo(BrokerPartitionInfo.scala:82)
at kafka.producer.BrokerPartitionInfo.getBrokerPartitionInfo(BrokerPartitionInfo.scala:49)
at kafka.producer.async.DefaultEventHandler.kafka$producer$async$DefaultEventHandler$$getPartitionListForTopic(DefaultEventHandler.scala:186)
at kafka.producer.async.DefaultEventHandler$$anonfun$partitionAndCollate$1.apply(DefaultEventHandler.scala:150)
at kafka.producer.async.DefaultEventHandler$$anonfun$partitionAndCollate$1.apply(DefaultEventHandler.scala:149)
at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:47)
at kafka.producer.async.DefaultEventHandler.partitionAndCollate(DefaultEventHandler.scala:149)
at kafka.producer.async.DefaultEventHandler.dispatchSerializedData(DefaultEventHandler.scala:95)
at kafka.producer.async.DefaultEventHandler.handle(DefaultEventHandler.scala:72)
at kafka.producer.async.ProducerSendThread.tryToHandle(ProducerSendThread.scala:105)
at kafka.producer.async.ProducerSendThread$$anonfun$processEvents$3.apply(ProducerSendThread.scala:88)
at kafka.producer.async.ProducerSendThread$$anonfun$processEvents$3.apply(ProducerSendThread.scala:68)
at scala.collection.immutable.Stream.foreach(Stream.scala:547)
at kafka.producer.async.ProducerSendThread.processEvents(ProducerSendThread.scala:67)
at kafka.producer.async.ProducerSendThread.run(ProducerSendThread.scala:45)
[2015-02-25 05:27:53,511] ERROR Failed to collate messages by topic, partition due to: fetching topic metadata for topics [Set(test)] from broker [ArrayBuffer(id:0,host:192.168.1.76,port:9092)] failed (kafka.producer.async.DefaultEventHandler)
[2015-02-25 05:27:54,614] WARN Fetching topic metadata with correlation id 2 for topics [Set(test)] from broker [id:0,host:192.168.1.76,port:9092] failed (kafka.client.ClientUtils$)
java.nio.channels.ClosedChannelException
at kafka.network.BlockingChannel.send(BlockingChannel.scala:100)
at kafka.producer.SyncProducer.liftedTree1$1(SyncProducer.scala:73)
at kafka.producer.SyncProducer.kafka$producer$SyncProducer$$doSend(SyncProducer.scala:72)
at kafka.producer.SyncProducer.send(SyncProducer.scala:113)
at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:58)
at kafka.producer.BrokerPartitionInfo.updateInfo(BrokerPartitionInfo.scala:82)
at kafka.producer.async.DefaultEventHandler$$anonfun$handle$2.apply$mcV$sp(DefaultEventHandler.scala:78)
at kafka.utils.Utils$.swallow(Utils.scala:172)
at kafka.utils.Logging$class.swallowError(Logging.scala:106)
at kafka.utils.Utils$.swallowError(Utils.scala:45)
at kafka.producer.async.DefaultEventHandler.handle(DefaultEventHandler.scala:78)
at kafka.producer.async.ProducerSendThread.tryToHandle(ProducerSendThread.scala:105)
at kafka.producer.async.ProducerSendThread$$anonfun$processEvents$3.apply(ProducerSendThread.scala:88)
at kafka.producer.async.ProducerSendThread$$anonfun$processEvents$3.apply(ProducerSendThread.scala:68)
at scala.collection.immutable.Stream.foreach(Stream.scala:547)
at kafka.producer.async.ProducerSendThread.processEvents(ProducerSendThread.scala:67)
at kafka.producer.async.ProducerSendThread.run(ProducerSendThread.scala:45)
[2015-02-25 05:27:54,615] ERROR fetching topic metadata for topics [Set(test)] from broker [ArrayBuffer(id:0,host:192.168.1.76,port:9092)] failed (kafka.utils.Utils$)
kafka.common.KafkaException: fetching topic metadata for topics [Set(test)] from broker [ArrayBuffer(id:0,host:192.168.1.76,port:9092)] failed
at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:72)
at kafka.producer.BrokerPartitionInfo.updateInfo(BrokerPartitionInfo.scala:82)
at kafka.producer.async.DefaultEventHandler$$anonfun$handle$2.apply$mcV$sp(DefaultEventHandler.scala:78)
at kafka.utils.Utils$.swallow(Utils.scala:172)
at kafka.utils.Logging$class.swallowError(Logging.scala:106)
at kafka.utils.Utils$.swallowError(Utils.scala:45)
at kafka.producer.async.DefaultEventHandler.handle(DefaultEventHandler.scala:78)
at kafka.producer.async.ProducerSendThread.tryToHandle(ProducerSendThread.scala:105)
at kafka.producer.async.ProducerSendThread$$anonfun$processEvents$3.apply(ProducerSendThread.scala:88)
at kafka.producer.async.ProducerSendThread$$anonfun$processEvents$3.apply(ProducerSendThread.scala:68)
at scala.collection.immutable.Stream.foreach(Stream.scala:547)
at kafka.producer.async.ProducerSendThread.processEvents(ProducerSendThread.scala:67)
at kafka.producer.async.ProducerSendThread.run(ProducerSendThread.scala:45)
Caused by: java.nio.channels.ClosedChannelException
at kafka.network.BlockingChannel.send(BlockingChannel.scala:100)
at kafka.producer.SyncProducer.liftedTree1$1(SyncProducer.scala:73)
at kafka.producer.SyncProducer.kafka$producer$SyncProducer$$doSend(SyncProducer.scala:72)
at kafka.producer.SyncProducer.send(SyncProducer.scala:113)
at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:58)
... 12 more
[2015-02-25 05:27:55,618] WARN Fetching topic metadata with correlation id 3 for topics [Set(test)] from broker [id:0,host:192.168.1.76,port:9092] failed (kafka.client.ClientUtils$)
java.nio.channels.ClosedChannelException
at kafka.network.BlockingChannel.send(BlockingChannel.scala:100)
at kafka.producer.SyncProducer.liftedTree1$1(SyncProducer.scala:73)
at kafka.producer.SyncProducer.kafka$producer$SyncProducer$$doSend(SyncProducer.scala:72)
at kafka.producer.SyncProducer.send(SyncProducer.scala:113)
at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:58)
at kafka.producer.BrokerPartitionInfo.updateInfo(BrokerPartitionInfo.scala:82)
at kafka.producer.BrokerPartitionInfo.getBrokerPartitionInfo(BrokerPartitionInfo.scala:49)
at kafka.producer.async.DefaultEventHandler.kafka$producer$async$DefaultEventHandler$$getPartitionListForTopic(DefaultEventHandler.scala:186)
at kafka.producer.async.DefaultEventHandler$$anonfun$partitionAndCollate$1.apply(DefaultEventHandler.scala:150)
at kafka.producer.async.DefaultEventHandler$$anonfun$partitionAndCollate$1.apply(DefaultEventHandler.scala:149)
at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:47)
at kafka.producer.async.DefaultEventHandler.partitionAndCollate(DefaultEventHandler.scala:149)
at kafka.producer.async.DefaultEventHandler.dispatchSerializedData(DefaultEventHandler.scala:95)
at kafka.producer.async.DefaultEventHandler.handle(DefaultEventHandler.scala:72)
at kafka.producer.async.ProducerSendThread.tryToHandle(ProducerSendThread.scala:105)
at kafka.producer.async.ProducerSendThread$$anonfun$processEvents$3.apply(ProducerSendThread.scala:88)
at kafka.producer.async.ProducerSendThread$$anonfun$processEvents$3.apply(ProducerSendThread.scala:68)
at scala.collection.immutable.Stream.foreach(Stream.scala:547)
at kafka.producer.async.ProducerSendThread.processEvents(ProducerSendThread.scala:67)
at kafka.producer.async.ProducerSendThread.run(ProducerSendThread.scala:45)
[2015-02-25 05:27:55,619] ERROR Failed to collate messages by topic, partition due to: fetching topic metadata for topics [Set(test)] from broker [ArrayBuffer(id:0,host:192.168.1.76,port:9092)] failed (kafka.producer.async.DefaultEventHandler)
[2015-02-25 05:27:56,722] WARN Fetching topic metadata with correlation id 4 for topics [Set(test)] from broker [id:0,host:192.168.1.76,port:9092] failed (kafka.client.ClientUtils$)
java.nio.channels.ClosedChannelException
at kafka.network.BlockingChannel.send(BlockingChannel.scala:100)
at kafka.producer.SyncProducer.liftedTree1$1(SyncProducer.scala:73)
at kafka.producer.SyncProducer.kafka$producer$SyncProducer$$doSend(SyncProducer.scala:72)
at kafka.producer.SyncProducer.send(SyncProducer.scala:113)
at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:58)
at kafka.producer.BrokerPartitionInfo.updateInfo(BrokerPartitionInfo.scala:82)
at kafka.producer.async.DefaultEventHandler$$anonfun$handle$2.apply$mcV$sp(DefaultEventHandler.scala:78)
at kafka.utils.Utils$.swallow(Utils.scala:172)
at kafka.utils.Logging$class.swallowError(Logging.scala:106)
at kafka.utils.Utils$.swallowError(Utils.scala:45)
at kafka.producer.async.DefaultEventHandler.handle(DefaultEventHandler.scala:78)
at kafka.producer.async.ProducerSendThread.tryToHandle(ProducerSendThread.scala:105)
at kafka.producer.async.ProducerSendThread$$anonfun$processEvents$3.apply(ProducerSendThread.scala:88)
at kafka.producer.async.ProducerSendThread$$anonfun$processEvents$3.apply(ProducerSendThread.scala:68)
at scala.collection.immutable.Stream.foreach(Stream.scala:547)
at kafka.producer.async.ProducerSendThread.processEvents(ProducerSendThread.scala:67)
at kafka.producer.async.ProducerSendThread.run(ProducerSendThread.scala:45)
[2015-02-25 05:27:56,723] ERROR fetching topic metadata for topics [Set(test)] from broker [ArrayBuffer(id:0,host:192.168.1.76,port:9092)] failed (kafka.utils.Utils$)
kafka.common.KafkaException: fetching topic metadata for topics [Set(test)] from broker [ArrayBuffer(id:0,host:192.168.1.76,port:9092)] failed
at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:72)
at kafka.producer.BrokerPartitionInfo.updateInfo(BrokerPartitionInfo.scala:82)
at kafka.producer.async.DefaultEventHandler$$anonfun$handle$2.apply$mcV$sp(DefaultEventHandler.scala:78)
at kafka.utils.Utils$.swallow(Utils.scala:172)
at kafka.utils.Logging$class.swallowError(Logging.scala:106)
at kafka.utils.Utils$.swallowError(Utils.scala:45)
at kafka.producer.async.DefaultEventHandler.handle(DefaultEventHandler.scala:78)
at kafka.producer.async.ProducerSendThread.tryToHandle(ProducerSendThread.scala:105)
at kafka.producer.async.ProducerSendThread$$anonfun$processEvents$3.apply(ProducerSendThread.scala:88)
at kafka.producer.async.ProducerSendThread$$anonfun$processEvents$3.apply(ProducerSendThread.scala:68)
at scala.collection.immutable.Stream.foreach(Stream.scala:547)
at kafka.producer.async.ProducerSendThread.processEvents(ProducerSendThread.scala:67)
at kafka.producer.async.ProducerSendThread.run(ProducerSendThread.scala:45)
Caused by: java.nio.channels.ClosedChannelException
at kafka.network.BlockingChannel.send(BlockingChannel.scala:100)
at kafka.producer.SyncProducer.liftedTree1$1(SyncProducer.scala:73)
at kafka.producer.SyncProducer.kafka$producer$SyncProducer$$doSend(SyncProducer.scala:72)
at kafka.producer.SyncProducer.send(SyncProducer.scala:113)
at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:58)
... 12 more
[2015-02-25 05:27:57,726] WARN Fetching topic metadata with correlation id 5 for topics [Set(test)] from broker [id:0,host:192.168.1.76,port:9092] failed (kafka.client.ClientUtils$)
java.nio.channels.ClosedChannelException
at kafka.network.BlockingChannel.send(BlockingChannel.scala:100)
at kafka.producer.SyncProducer.liftedTree1$1(SyncProducer.scala:73)
at kafka.producer.SyncProducer.kafka$producer$SyncProducer$$doSend(SyncProducer.scala:72)
at kafka.producer.SyncProducer.send(SyncProducer.scala:113)
at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:58)
at kafka.producer.BrokerPartitionInfo.updateInfo(BrokerPartitionInfo.scala:82)
at kafka.producer.BrokerPartitionInfo.getBrokerPartitionInfo(BrokerPartitionInfo.scala:49)
at kafka.producer.async.DefaultEventHandler.kafka$producer$async$DefaultEventHandler$$getPartitionListForTopic(DefaultEventHandler.scala:186)
at kafka.producer.async.DefaultEventHandler$$anonfun$partitionAndCollate$1.apply(DefaultEventHandler.scala:150)
at kafka.producer.async.DefaultEventHandler$$anonfun$partitionAndCollate$1.apply(DefaultEventHandler.scala:149)
at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:47)
at kafka.producer.async.DefaultEventHandler.partitionAndCollate(DefaultEventHandler.scala:149)
at kafka.producer.async.DefaultEventHandler.dispatchSerializedData(DefaultEventHandler.scala:95)
at kafka.producer.async.DefaultEventHandler.handle(DefaultEventHandler.scala:72)
at kafka.producer.async.ProducerSendThread.tryToHandle(ProducerSendThread.scala:105)
at kafka.producer.async.ProducerSendThread$$anonfun$processEvents$3.apply(ProducerSendThread.scala:88)
at kafka.producer.async.ProducerSendThread$$anonfun$processEvents$3.apply(ProducerSendThread.scala:68)
at scala.collection.immutable.Stream.foreach(Stream.scala:547)
at kafka.producer.async.ProducerSendThread.processEvents(ProducerSendThread.scala:67)
at kafka.producer.async.ProducerSendThread.run(ProducerSendThread.scala:45)
[2015-02-25 05:27:57,727] ERROR Failed to collate messages by topic, partition due to: fetching topic metadata for topics [Set(test)] from broker [ArrayBuffer(id:0,host:192.168.1.76,port:9092)] failed (kafka.producer.async.DefaultEventHandler)
^C[2015-02-25 05:27:58,830] WARN Fetching topic metadata with correlation id 6 for topics [Set(test)] from broker [id:0,host:192.168.1.76,port:9092] failed (kafka.client.ClientUtils$)
java.nio.channels.ClosedChannelException
at kafka.network.BlockingChannel.send(BlockingChannel.scala:100)
at kafka.producer.SyncProducer.liftedTree1$1(SyncProducer.scala:73)
at kafka.producer.SyncProducer.kafka$producer$SyncProducer$$doSend(SyncProducer.scala:72)
at kafka.producer.SyncProducer.send(SyncProducer.scala:113)
at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:58)
at kafka.producer.BrokerPartitionInfo.updateInfo(BrokerPartitionInfo.scala:82)
at kafka.producer.async.DefaultEventHandler$$anonfun$handle$2.apply$mcV$sp(DefaultEventHandler.scala:78)
at kafka.utils.Utils$.swallow(Utils.scala:172)
at kafka.utils.Logging$class.swallowError(Logging.scala:106)
at kafka.utils.Utils$.swallowError(Utils.scala:45)
at kafka.producer.async.DefaultEventHandler.handle(DefaultEventHandler.scala:78)
at kafka.producer.async.ProducerSendThread.tryToHandle(ProducerSendThread.scala:105)
at kafka.producer.async.ProducerSendThread$$anonfun$processEvents$3.apply(ProducerSendThread.scala:88)
at kafka.producer.async.ProducerSendThread$$anonfun$processEvents$3.apply(ProducerSendThread.scala:68)
at scala.collection.immutable.Stream.foreach(Stream.scala:547)
at kafka.producer.async.ProducerSendThread.processEvents(ProducerSendThread.scala:67)
at kafka.producer.async.ProducerSendThread.run(ProducerSendThread.scala:45)
[2015-02-25 05:27:58,831] ERROR fetching topic metadata for topics [Set(test)] from broker [ArrayBuffer(id:0,host:192.168.1.76,port:9092)] failed (kafka.utils.Utils$)
kafka.common.KafkaException: fetching topic metadata for topics [Set(test)] from broker [ArrayBuffer(id:0,host:192.168.1.76,port:9092)] failed
at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:72)
at kafka.producer.BrokerPartitionInfo.updateInfo(BrokerPartitionInfo.scala:82)
at kafka.producer.async.DefaultEventHandler$$anonfun$handle$2.apply$mcV$sp(DefaultEventHandler.scala:78)
at kafka.utils.Utils$.swallow(Utils.scala:172)
at kafka.utils.Logging$class.swallowError(Logging.scala:106)
at kafka.utils.Utils$.swallowError(Utils.scala:45)
at kafka.producer.async.DefaultEventHandler.handle(DefaultEventHandler.scala:78)
at kafka.producer.async.ProducerSendThread.tryToHandle(ProducerSendThread.scala:105)
at kafka.producer.async.ProducerSendThread$$anonfun$processEvents$3.apply(ProducerSendThread.scala:88)
at kafka.producer.async.ProducerSendThread$$anonfun$processEvents$3.apply(ProducerSendThread.scala:68)
at scala.collection.immutable.Stream.foreach(Stream.scala:547)
at kafka.producer.async.ProducerSendThread.processEvents(ProducerSendThread.scala:67)
at kafka.producer.async.ProducerSendThread.run(ProducerSendThread.scala:45)
Caused by: java.nio.channels.ClosedChannelException
at kafka.network.BlockingChannel.send(BlockingChannel.scala:100)
at kafka.producer.SyncProducer.liftedTree1$1(SyncProducer.scala:73)
at kafka.producer.SyncProducer.kafka$producer$SyncProducer$$doSend(SyncProducer.scala:72)
at kafka.producer.SyncProducer.send(SyncProducer.scala:113)
at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:58)
... 12 more
[2015-02-25 05:27:59,834] WARN Fetching topic metadata with correlation id 7 for topics [Set(test)] from broker [id:0,host:192.168.1.76,port:9092] failed (kafka.client.ClientUtils$)
java.nio.channels.ClosedChannelException
at kafka.network.BlockingChannel.send(BlockingChannel.scala:100)
at kafka.producer.SyncProducer.liftedTree1$1(SyncProducer.scala:73)
at kafka.producer.SyncProducer.kafka$producer$SyncProducer$$doSend(SyncProducer.scala:72)
at kafka.producer.SyncProducer.send(SyncProducer.scala:113)
at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:58)
at kafka.producer.BrokerPartitionInfo.updateInfo(BrokerPartitionInfo.scala:82)
at kafka.producer.BrokerPartitionInfo.getBrokerPartitionInfo(BrokerPartitionInfo.scala:49)
at kafka.producer.async.DefaultEventHandler.kafka$producer$async$DefaultEventHandler$$getPartitionListForTopic(DefaultEventHandler.scala:186)
at kafka.producer.async.DefaultEventHandler$$anonfun$partitionAndCollate$1.apply(DefaultEventHandler.scala:150)
at kafka.producer.async.DefaultEventHandler$$anonfun$partitionAndCollate$1.apply(DefaultEventHandler.scala:149)
at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:47)
at kafka.producer.async.DefaultEventHandler.partitionAndCollate(DefaultEventHandler.scala:149)
at kafka.producer.async.DefaultEventHandler.dispatchSerializedData(DefaultEventHandler.scala:95)
at kafka.producer.async.DefaultEventHandler.handle(DefaultEventHandler.scala:72)
at kafka.producer.async.ProducerSendThread.tryToHandle(ProducerSendThread.scala:105)
at kafka.producer.async.ProducerSendThread$$anonfun$processEvents$3.apply(ProducerSendThread.scala:88)
at kafka.producer.async.ProducerSendThread$$anonfun$processEvents$3.apply(ProducerSendThread.scala:68)
at scala.collection.immutable.Stream.foreach(Stream.scala:547)
at kafka.producer.async.ProducerSendThread.processEvents(ProducerSendThread.scala:67)
at kafka.producer.async.ProducerSendThread.run(ProducerSendThread.scala:45)
[2015-02-25 05:27:59,835] ERROR Failed to collate messages by topic, partition due to: fetching topic metadata for topics [Set(test)] from broker [ArrayBuffer(id:0,host:192.168.1.76,port:9092)] failed (kafka.producer.async.DefaultEventHandler)
^C[2015-02-25 05:28:00,938] WARN Fetching topic metadata with correlation id 8 for topics [Set(test)] from broker [id:0,host:192.168.1.76,port:9092] failed (kafka.client.ClientUtils$)
java.nio.channels.ClosedChannelException
at kafka.network.BlockingChannel.send(BlockingChannel.scala:100)
at kafka.producer.SyncProducer.liftedTree1$1(SyncProducer.scala:73)
at kafka.producer.SyncProducer.kafka$producer$SyncProducer$$doSend(SyncProducer.scala:72)
at kafka.producer.SyncProducer.send(SyncProducer.scala:113)
at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:58)
at kafka.producer.BrokerPartitionInfo.updateInfo(BrokerPartitionInfo.scala:82)
at kafka.producer.async.DefaultEventHandler$$anonfun$handle$2.apply$mcV$sp(DefaultEventHandler.scala:78)
at kafka.utils.Utils$.swallow(Utils.scala:172)
at kafka.utils.Logging$class.swallowError(Logging.scala:106)
at kafka.utils.Utils$.swallowError(Utils.scala:45)
at kafka.producer.async.DefaultEventHandler.handle(DefaultEventHandler.scala:78)
at kafka.producer.async.ProducerSendThread.tryToHandle(ProducerSendThread.scala:105)
at kafka.producer.async.ProducerSendThread$$anonfun$processEvents$3.apply(ProducerSendThread.scala:88)
at kafka.producer.async.ProducerSendThread$$anonfun$processEvents$3.apply(ProducerSendThread.scala:68)
at scala.collection.immutable.Stream.foreach(Stream.scala:547)
at kafka.producer.async.ProducerSendThread.processEvents(ProducerSendThread.scala:67)
at kafka.producer.async.ProducerSendThread.run(ProducerSendThread.scala:45)
[2015-02-25 05:28:00,939] ERROR fetching topic metadata for topics [Set(test)] from broker [ArrayBuffer(id:0,host:192.168.1.76,port:9092)] failed (kafka.utils.Utils$)
kafka.common.KafkaException: fetching topic metadata for topics [Set(test)] from broker [ArrayBuffer(id:0,host:192.168.1.76,port:9092)] failed
at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:72)
at kafka.producer.BrokerPartitionInfo.updateInfo(BrokerPartitionInfo.scala:82)
at kafka.producer.async.DefaultEventHandler$$anonfun$handle$2.apply$mcV$sp(DefaultEventHandler.scala:78)
at kafka.utils.Utils$.swallow(Utils.scala:172)
at kafka.utils.Logging$class.swallowError(Logging.scala:106)
at kafka.utils.Utils$.swallowError(Utils.scala:45)
at kafka.producer.async.DefaultEventHandler.handle(DefaultEventHandler.scala:78)
at kafka.producer.async.ProducerSendThread.tryToHandle(ProducerSendThread.scala:105)
at kafka.producer.async.ProducerSendThread$$anonfun$processEvents$3.apply(ProducerSendThread.scala:88)
at kafka.producer.async.ProducerSendThread$$anonfun$processEvents$3.apply(ProducerSendThread.scala:68)
at scala.collection.immutable.Stream.foreach(Stream.scala:547)
at kafka.producer.async.ProducerSendThread.processEvents(ProducerSendThread.scala:67)
at kafka.producer.async.ProducerSendThread.run(ProducerSendThread.scala:45)
Caused by: java.nio.channels.ClosedChannelException
at kafka.network.BlockingChannel.send(BlockingChannel.scala:100)
at kafka.producer.SyncProducer.liftedTree1$1(SyncProducer.scala:73)
at kafka.producer.SyncProducer.kafka$producer$SyncProducer$$doSend(SyncProducer.scala:72)
at kafka.producer.SyncProducer.send(SyncProducer.scala:113)
at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:58)
... 12 more
[2015-02-25 05:28:00,940] ERROR Failed to send requests for topics test with correlation ids in [0,8] (kafka.producer.async.DefaultEventHandler)
[2015-02-25 05:28:00,941] ERROR Error in handling batch of 1 events (kafka.producer.async.ProducerSendThread)
kafka.common.FailedToSendMessageException: Failed to send messages after 3 tries.
at kafka.producer.async.DefaultEventHandler.handle(DefaultEventHandler.scala:90)
at kafka.producer.async.ProducerSendThread.tryToHandle(ProducerSendThread.scala:105)
at kafka.producer.async.ProducerSendThread$$anonfun$processEvents$3.apply(ProducerSendThread.scala:88)
at kafka.producer.async.ProducerSendThread$$anonfun$processEvents$3.apply(ProducerSendThread.scala:68)
at scala.collection.immutable.Stream.foreach(Stream.scala:547)
at kafka.producer.async.ProducerSendThread.processEvents(ProducerSendThread.scala:67)
at kafka.producer.async.ProducerSendThread.run(ProducerSendThread.scala:45)
root@419a95c35482:/#
zookeeper (obviously broken)
➜ kafka-docker git:(master) ✗ ./start-kafka-shell.sh 192.168.1.76 192.168.1.76:2181
root@b42808961755:/# $KAFKA_HOME/bin/zookeeper-shell.sh 192.168.1.76:2181
Connecting to 192.168.1.76:2181
Welcome to ZooKeeper!
JLine support is disabled
[2015-02-25 05:30:54,611] WARN Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect (org.apache.zookeeper.ClientCnxn)
java.net.NoRouteToHostException: No route to host
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:601)
at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)
[2015-02-25 05:30:56,718] WARN Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect (org.apache.zookeeper.ClientCnxn)
java.net.NoRouteToHostException: No route to host
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:601)
at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)
^Croot@b42808961755:/#
can you please try the following:
docker run -ti busybox ping -c 2 192.168.1.76
docker run -ti busybox ip -4 route show
on your docker host (192.168.1.76) start nc -l 9999 to listen on port 9999.
docker run -ti busybox telnet 192.168.1.76 9999
If this runs successfully, you should be able to type anything into your shell and see the result in the terminal you started nc on.
Please let me know the results of these tests.
Thanks
first telnet window:
➜ kafka-docker git:(master) ✗ docker run -ti busybox ping -c 2 192.168.1.76
PING 192.168.1.76 (192.168.1.76): 56 data bytes
64 bytes from 192.168.1.76: seq=0 ttl=64 time=0.079 ms
64 bytes from 192.168.1.76: seq=1 ttl=64 time=0.083 ms
--- 192.168.1.76 ping statistics ---
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max = 0.079/0.081/0.083 ms
➜ kafka-docker git:(master) ✗ docker run -ti busybox ip -4 route show
default via 172.17.42.1 dev eth0
172.17.0.0/16 dev eth0 proto kernel scope link src 172.17.0.9
➜ kafka-docker git:(master) ✗
➜ kafka-docker git:(master) ✗ nc -l 9999
➜ kafka-docker git:(master) ✗
second telnet window:
➜ dev docker run -ti busybox telnet 192.168.1.76 9999
telnet: can't connect to remote host (192.168.1.76): No route to host
➜ dev
That's definitely odd, this laptop is fedora 21 and recently installed with not much modification.
just confirming the same thing:
➜ ~ docker run -i -t ubuntu /bin/bash
root@51077a9edf6c:/# telnet 192.168.1.76 9999
bash: telnet: command not found
root@51077a9edf6c:/# apt-get install telnet
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following NEW packages will be installed:
telnet
0 upgraded, 1 newly installed, 0 to remove and 0 not upgraded.
Need to get 67.1 kB of archives.
After this operation, 167 kB of additional disk space will be used.
Get:1 http://archive.ubuntu.com/ubuntu/ trusty/main telnet amd64 0.17-36build2 [67.1 kB]
Fetched 67.1 kB in 0s (90.6 kB/s) y
Selecting previously unselected package telnet.
(Reading database ... 11527 files and directories currently installed.)
Preparing to unpack .../telnet_0.17-36build2_amd64.deb ...
Unpacking telnet (0.17-36build2) ...
Setting up telnet (0.17-36build2) ...
update-alternatives: using /usr/bin/telnet.netkit to provide /usr/bin/telnet (telnet) in auto mode
root@51077a9edf6c:/# telnet 192.168.1.76 9999
Trying 192.168.1.76...
telnet: Unable to connect to remote host: No route to host
root@51077a9edf6c:/# ip -4 route show
default via 172.17.42.1 dev eth0
172.17.0.0/16 dev eth0 proto kernel scope link src 172.17.0.14
root@51077a9edf6c:/# ping 192.168.1.76
PING 192.168.1.76 (192.168.1.76) 56(84) bytes of data.
64 bytes from 192.168.1.76: icmp_seq=1 ttl=64 time=0.054 ms
64 bytes from 192.168.1.76: icmp_seq=2 ttl=64 time=0.055 ms
--- 192.168.1.76 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 999ms
rtt min/avg/max/mdev = 0.054/0.054/0.055/0.007 ms
my docker networking is obviously hosed, just experimenting with adding routes to fix it.
It's the firewall.
Crap, apologies and thanks.
➜ kafka-docker git:(master) sudo systemctl stop firewalld.service
➜ kafka-docker git:(master) ✗ nc -l 9999
from ubuntu box:
root@51077a9edf6c:/# telnet 192.168.1.76 9999
Trying 192.168.1.76...
Connected to 192.168.1.76.
Escape character is '^]'.
Connection closed by foreign host.
Appears so, thanks very much.
Relevant details:
➜ kafka-docker git:(master) ✗ sudo systemctl status firewalld.service
● firewalld.service - firewalld - dynamic firewall daemon
Loaded: loaded (/usr/lib/systemd/system/firewalld.service; enabled)
Active: active (running) since Sun 2015-03-01 15:36:23 PST; 1h 30min ago
Main PID: 863 (firewalld)
CGroup: /system.slice/firewalld.service
└─863 /usr/bin/python -Es /usr/sbin/firewalld --nofork --nopid
Mar 01 15:37:34 gclaptop firewalld[863]: 2015-03-01 15:37:34 ERROR: UNKNOWN_INTERFACE: ...one
Hint: Some lines were ellipsized, use -l to show in full.
➜ kafka-docker git:(master) ✗ sudo systemctl stop firewalld.service
➜ kafka-docker git:(master) ✗ sudo service docker start
Redirecting to /bin/systemctl start docker.service
➜ kafka-docker git:(master) ✗ fig ps
Name Command State Ports
------------------------------
➜ kafka-docker git:(master) ✗ fig -f fig-single-broker.yml up -d
Creating kafkadocker_zookeeper_1...
Creating kafkadocker_kafka_1...
➜ kafka-docker git:(master) ✗
testing producer:
➜ kafka_2.9.2-0.8.2.0 ls
bin config libs LICENSE logs NOTICE
➜ kafka_2.9.2-0.8.2.0 bin/kafka-topics.sh --list --zookeeper 192.168.1.76:2181
➜ kafka_2.9.2-0.8.2.0 bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic test
Created topic "test".
➜ kafka_2.9.2-0.8.2.0 bin/kafka-topics.sh --list --zookeeper 192.168.1.76:2181
➜ kafka_2.9.2-0.8.2.0 bin/kafka-console-producer.sh --broker-list localhost:9092 --topic test
[2015-03-01 17:09:37,634] WARN Property topic is not valid (kafka.utils.VerifiableProperties)
dederp
derp derp derp
testing consumer:
➜ kafka_2.9.2-0.8.2.0 bin/kafka-console-consumer.sh --zookeeper localhost:2181 --topic test --from-beginning
dederp
derp derp derp
I also found some relevant bugs:
https://bugzilla.redhat.com/show_bug.cgi?id=1033606
I'll try to find something more recent and contribute there.
Thanks again.
Got 'org.apache.kafka.common.errors.TimeoutException' when send message on linux Fedora25/CentOS7
推荐文章
|
|
乖乖的蜡烛 · View Source 11 小时前 |
|
|
愉快的铁链 · 30个Kafka常见错误小集合-腾讯云开发者社区-腾讯云 11 小时前 |
|
|
温文尔雅的紫菜汤 · WARN org.apache.kafka.clients.NetworkClient - Bootstrap broker 127.0.0.1:9092 disconnected_bootstrap 11 小时前 |
|
|
沉着的脸盆 · 软件： ESPRIT 系列 | Bruker 11 月前 |