authorizer.class.name=kafka.security.authorizer.AclAuthorizer
Kafka 3.0 remove SimpleAclAuthorizer
michael
security.inter.broker.protocol=SASL_PLAINTEXT
sasl.mechanism.inter.broker.protocol=PLAIN
sasl.enabled.mechanisms=PLAIN
authorizer.class.name=kafka.security.auth.AclAuthorizer
allow.everyone.if.no.acl.found=true
listeners=SASL_PLAINTEXT://10.119.37.108:9092
advertised.listeners=SASL_PLAINTEXT://10.119.37.108:9092
Error message -
[2022-07-27 14:50:15,407] INFO Registered kafka:type=kafka.Log4jController MBean (kafka.utils.Log4jControllerRegistration$)
[2022-07-27 14:50:15,695] INFO Setting -D jdk.tls.rejectClientInitiatedRenegotiation=true to disable client-initiated TLS renegotiation (org.apache.zookeeper.common.X509Util)
[2022-07-27 14:50:15,707] ERROR Exiting Kafka due to fatal exception (kafka.Kafka$)
java.lang.ClassNotFoundException: kafka.security.auth.AclAuthorizer
at java.net.URLClassLoader.findClass(URLClassLoader.java:387)
at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:348)
at org.apache.kafka.common.utils.Utils.loadClass(Utils.java:419)
at org.apache.kafka.common.utils.Utils.newInstance(Utils.java:408)
at kafka.security.authorizer.AuthorizerUtils$.createAuthorizer(AuthorizerUtils.scala:31)
at kafka.server.KafkaConfig.(KafkaConfig.scala:1658)
at kafka.server.KafkaConfig.(KafkaConfig.scala:1471)
at kafka.Kafka$.buildServer(Kafka.scala:67)
at kafka.Kafka$.main(Kafka.scala:87)
at kafka.Kafka.main(Kafka.scala)
I have executed below 2 commands after updating above mentioned parameters.
export KAFKA_OPTS=“-Djava.security.auth.login.config=/opt/kafka/config/zookeeper_jaas.conf”
export KAFKA_OPTS=“-Djava.security.auth.login.config=/opt/kafka/config/kafka_server_jaas.conf”
WIll these commands creating any problem ?
ok I see
classpath is set correctly
you’re using apache kafka not the confluent one, correct?
best,
michael
just to be sure
parameter looks like:
authorizer.class.name=kafka.security.authorizer.AclAuthorizer
right?
as the log is complaining about kafka.security.auth.AclAuthorizer
best,
michael
HI Michael,
Now old issue is resolved after changing the above line in server.property file.
Yes i am using apche kafka verison.
now getting new error.
[2022-07-27 16:33:13,637] INFO Completed load of Log(dir=/tmp/kafka-logs-new/applogmessages-0, topicId=iR3Hl7HhTwS-QRF5zNv40g, topic=applogmessages, partition=0, highWatermark=0, lastStableOffset=0, logStartOffset=0, logEndOffset=0) with 1 segments in 3ms (52/52 loaded in /tmp/kafka-logs-new) (kafka.log.LogManager)
[2022-07-27 16:33:13,638] INFO Loaded 52 logs in 340ms. (kafka.log.LogManager)
[2022-07-27 16:33:13,639] INFO Starting log cleanup with a period of 300000 ms. (kafka.log.LogManager)
[2022-07-27 16:33:13,639] INFO Starting log flusher with a default period of 9223372036854775807 ms. (kafka.log.LogManager)
[2022-07-27 16:33:13,901] ERROR [KafkaServer id=18] Fatal error during KafkaServer startup. Prepare to shutdown (kafka.server.KafkaServer)
java.lang.IllegalArgumentException: Could not find a 'KafkaServer' or 'sasl_plaintext.KafkaServer' entry in the JAAS configuration. System property 'java.security.auth.login.config' is /opt/kafka/config/zookeeper_jaas.conf
at org.apache.kafka.common.security.JaasContext.defaultContext(JaasContext.java:131)
at org.apache.kafka.common.security.JaasContext.load(JaasContext.java:96)
at org.apache.kafka.common.security.JaasContext.loadServerContext(JaasContext.java:69)
at org.apache.kafka.common.network.ChannelBuilders.create(ChannelBuilders.java:168)
at org.apache.kafka.common.network.ChannelBuilders.clientChannelBuilder(ChannelBuilders.java:81)
at kafka.server.BrokerToControllerChannelManagerImpl.newRequestThread(BrokerToControllerChannelManager.scala:189)
at kafka.server.BrokerToControllerChannelManagerImpl.<init>(BrokerToControllerChannelManager.scala:168)
at kafka.server.KafkaServer.startup(KafkaServer.scala:284)
at kafka.Kafka$.main(Kafka.scala:109)
at kafka.Kafka.main(Kafka.scala)
[2022-07-27 16:33:13,903] INFO [KafkaServer id=18] shutting down (kafka.server.KafkaServer)
[2022-07-27 16:33:13,906] INFO Shutting down. (kafka.log.LogManager)
[2022-07-27 16:33:14,003] INFO Shutdown complete. (kafka.log.LogManager)
[2022-07-27 16:33:14,003] INFO [feature-zk-node-event-process-thread]: Shutting down (kafka.server.FinalizedFeatureChangeListener$ChangeNotificationProcessorThread)
[2022-07-27 16:33:14,003] INFO [feature-zk-node-event-process-thread]: Stopped (kafka.server.FinalizedFeatureChangeListener$ChangeNotificationProcessorThread)
[2022-07-27 16:33:14,003] INFO [feature-zk-node-event-process-thread]: Shutdown completed (kafka.server.FinalizedFeatureChangeListener$ChangeNotificationProcessorThread)
[2022-07-27 16:33:14,004] INFO [ZooKeeperClient Kafka server] Closing. (kafka.zookeeper.ZooKeeperClient)
[2022-07-27 16:33:14,113] INFO Session: 0x100cd21b5650001 closed (org.apache.zookeeper.ZooKeeper)
[2022-07-27 16:33:14,113] INFO EventThread shut down for session: 0x100cd21b5650001 (org.apache.zookeeper.ClientCnxn)
Hi Michel,
One more observation…
I am running both zookeeper and Kafka server on same machine.
if i execute below command and then if i start the kafka server, getting below error
export KAFKA_OPTS="—Djava.security.auth.login.config=/opt/kafka/config/zookeeper_jaas.conf
Error
[2022-07-27 16:59:34,898] ERROR [KafkaServer id=18] Fatal error during KafkaServer startup. Prepare to shutdown (kafka.server.KafkaServer)
java.lang.IllegalArgumentException: Could not find a 'KafkaServer' or 'sasl_plaintext.KafkaServer' entry in the JAAS configuration. System property 'java.security.auth.login.config' is /opt/kafka/config/zookeeper_jaas.conf
at org.apache.kafka.common.security.JaasContext.defaultContext(JaasContext.java:131)
at org.apache.kafka.common.security.JaasContext.load(JaasContext.java:96)
at org.apache.kafka.common.security.JaasContext.loadServerContext(JaasContext.java:69)
at org.apache.kafka.common.network.ChannelBuilders.create(ChannelBuilders.java:168)
at org.apache.kafka.common.network.ChannelBuilders.clientChannelBuilder(ChannelBuilders.java:81)
at kafka.server.BrokerToControllerChannelManagerImpl.newRequestThread(BrokerToControllerChannelManager.scala:189)
at kafka.server.BrokerToControllerChannelManagerImpl.<init>(BrokerToControllerChannelManager.scala:168)
at kafka.server.KafkaServer.startup(KafkaServer.scala:284)
at kafka.Kafka$.main(Kafka.scala:109)
at kafka.Kafka.main(Kafka.scala)
[2022-07-27 16:59:34,900] INFO [KafkaServer id=18] shutting down (kafka.server.KafkaServer)
And if set below value , and start the kafka server, getting below error.
export KAFKA_OPTS=“-Djava.security.auth.login.config=/opt/kafka/config/kafka_server_jaas.conf”
error message -
[2022-07-27 17:01:02,376] ERROR SASL authentication failed using login context 'Client'. (org.apache.zookeeper.client.ZooKeeperSaslClient)
javax.security.sasl.SaslException: Error in authenticating with a Zookeeper Quorum member: the quorum member's saslToken is null.
[2022-07-27 17:01:02,383] INFO EventThread shut down for session: 0x100cd40db240001 (org.apache.zookeeper.ClientCnxn)
[2022-07-27 17:01:02,438] ERROR Fatal error during KafkaServer startup. Prepare to shutdown (kafka.server.KafkaServer)
org.apache.zookeeper.KeeperException$AuthFailedException: KeeperErrorCode = AuthFailed for /consumers
at org.apache.zookeeper.KeeperException.create(KeeperException.java:130)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:54)
at kafka.zookeeper.AsyncResponse.maybeThrow(ZooKeeperClient.scala:566)
How to set both the values in the same server…
export KAFKA_OPTS=“-Djava.security.auth.login.config=/opt/kafka/config/kafka_server_jaas.conf”
export KAFKA_OPTS=“-Djava.security.auth.login.config=/opt/kafka/config/zookeeper_jaas.conf”
hey @Harijld
I think the easiest way is to use 2 separate shell sessions.
or use 2 separate user if you’d like to separate in this way.
best,
michael
HI Michael,
I ran below 2 commands with .sh and now it is running but giving error while connecting to zookeeper.
export KAFKA_OPTS=-Djava.security.auth.login.config=/opt/kafka/config/kafka_server_jaas.conf
./bin/kafka-server-start.sh config/server.properties
export KAFKA_OPTS=-Djava.security.auth.login.config=/opt/kafka/config/zookeeper_jaas.conf
./bin/zookeeper-server-start.sh config/zookeeper.properties
Error message -
[2022-07-28 13:20:12,213] INFO [SocketServer listenerType=ZK_BROKER, nodeId=18] Failed authentication with /10.119.37.107 (channelId=10.xxx.xx.xxx:9092-10.xxx.xx.xxx:44958-3) (Unexpected Kafka request of type METADATA during SASL handshake.) (org.apache.kafka.common.network.Selector)
[2022-07-28 13:20:12,618] INFO [SocketServer listenerType=ZK_BROKER, nodeId=18] Failed authentication with /10.xxx.xx.xxx (channelId=10.xxx.xx.xxx:9092-10.xxx.xx.xxx:44960-3) (Unexpected Kafka request of type METADATA during SASL handshake.) (org.apache.kafka.common.network.Selector)
Some property / parameter issue at client or server end.
Hello. Reminder that this section of the forum is for the Kafka Connect API, not general topics of “connecting to Kafka”.
For that, Ops or Clients sections would be preferred.
