Kafka TLS/SSL encryption and Access Control Lists (ACLs) setup
Table of contents
- Preamble
- Kafka TLS/SSL encryption
- Access Control Lists (ACLs)
-
Issues encountered
- org.apache.kafka.common.errors.SslAuthenticationException: SSL handshake failed
- Error creating Kafka producer: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: Hostname mismatch, certificate is not valid for 'server01.domain.com'. (_ssl.c:1028)
- Error creating Kafka producer: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self-signed certificate in certificate chain (_ssl.c:1028)
- listenerName=ListenerName(CONTROLLER), securityProtocol=SASL_SSL, buffer=null, envelope=None) is not authorized
- References
Preamble
In my previous article (http) where I implemented SASL/PLAIN for my Kafka server I noted an important security warning in Confluent documentation:
SASL/PLAIN should only be used with TLS as transport layer to ensure that clear passwords are not transmitted on the wire without encryption.
Before going further in my testing in trying to give authorization to users this Data in Motion protection with TLS Encryption is a required step.
I also wanted to implement LDAP authentication but when I have tried to use the Confluent LDAP authentication:
listener.name.broker.plain.sasl.server.callback.handler.class = io.confluent.security.auth.provider.ldap.LdapAuthenticateCallbackHandler
I got:
Class io.confluent.security.auth.provider.ldap.LdapAuthenticateCallbackHandler could not be found.
The LDAP authentication is not in Confluent Community edition, you need the Enterprise one to implement it. There are few open source Kafka LDAP class or you can even write your own but this is a full project in itself…
Again I have struggled a lot in what was supposed to be a simple configuration to encrypt traffic between client and server and Confluent documentation is lacking a lot practical examples…
I have used Confluent Community Platform 7.9.1. My test server is running Red Hat Enterprise Linux release 8.10 (Ootpa) with 64GB of memory and 12 cores. On my Windows client I have used Python 3.13.3.
I have also used confluent-kafka 2.10.0 and kafka-python 2.2.7 Python libraries.
The high level big pictures is the following (diagrams generated with https://mermaid.js.org/ that is a really nice discover)

Kafka TLS/SSL encryption
It is strongly suggested to keep hostname verification for client connections as Confluent has written in documentation:
Host name verification of servers is enabled by default for client connections as well as interbroker connections to prevent man-in-the-middle attacks.
For testing purpose it is oaky to deactivate it but from security standpoint it is a must to keep it (particularly in production), set ssl.endpoint.identification.algorithm to an empty string in server.properties file:
ssl.endpoint.identification.algorithm=
Hostname verification can be done with two methods:
- Common Name (CN)
- Subject Alternative Name (SAN) (-ext SAN=DNS:hostname)
CN is relatively straightforward while the SAN one has kept me busy for multiple hours… It is strongly suggested to inject all parameters to avoid dying prompt of keytool and openssl (location, name, …). This is particularly true if, like me, you perform multiple tens of trials… Even if unsecure I am also using -storepass option of keytool to avoid typing my keystore password multiple tens times per day…
Kafka TLS/SSL with common Name (CN)
Generate the key and the certificate for each Confluent Server broker in the cluster (notice the CN=server01.domain.com that is your FQDN server name):
keytool -keystore kafka.server.keystore.jks -alias server01.domain.com -keyalg RSA -validity 730 -genkey -storepass secure_password -dname "CN=server01.domain.com, OU=IT, O=IT, L=Geneva, S=Geneva, C=CH"
Generate a Certificate Authority (CA) that is simply a public(ca.cert)-private(ca.key) key pair and certificate. To avoid answering question I have created this openssl.conf.cn file first (notice the commonName parameter):
[kafka@server01 ssl]$ cat openssl.conf.cn [ req ] default_bits = 2048 encrypt_key = no default_md = sha256 utf8 = yes string_mask = utf8only prompt = no distinguished_name = req_distinguished_name [ req_distinguished_name ] countryName = CH stateOrProvinceName = Geneva localityName = Geneva organizationName = MyCompany organizationalUnitName = IT commonName = server01.domain.com [kafka@server01 ssl]$ openssl req -new -x509 -keyout ca.key -out ca.cert -days 730 -config openssl.conf.cn
Store the CA in client and server truststore:
keytool -keystore kafka.client.truststore.jks -alias CARoot -importcert -file ca.cert -storepass secure_password -noprompt keytool -keystore kafka.server.truststore.jks -alias CARoot -importcert -file ca.cert -storepass secure_password -noprompt
Export the certificate from the keystore:
keytool -keystore kafka.server.keystore.jks -alias server01.domain.com -certreq -file cert-file -storepass secure_password -noprompt
Sign it with the CA:
openssl x509 -req -CA ca.cert -CAkey ca.key -in cert-file -out cert-signed -days 730 -CAcreateserial
Import both the certificate of the CA and the signed certificate into the broker keystore:
keytool -keystore kafka.server.keystore.jks -alias CARoot -importcert -file ca.cert -storepass secure_password -noprompt keytool -keystore kafka.server.keystore.jks -alias server01.domain.com -importcert -file cert-signed -storepass secure_password -noprompt
All in one in a generate.sh.cn script:
rm *.jks ca* cert* *.pem keytool -keystore kafka.server.keystore.jks -alias server01.domain.com -keyalg RSA -validity 730 -genkey -storepass secure_password -dname "CN=server01.domain.com, OU=IT, O=IT, L=Geneva, S=Geneva, C=CH" openssl req -new -x509 -keyout ca.key -out ca.cert -days 730 -config openssl.conf.cn keytool -keystore kafka.client.truststore.jks -alias CARoot -importcert -file ca.cert -storepass secure_password -noprompt keytool -keystore kafka.server.truststore.jks -alias CARoot -importcert -file ca.cert -storepass secure_password -noprompt keytool -keystore kafka.server.keystore.jks -alias server01.domain.com -certreq -file cert-file -storepass secure_password -noprompt openssl x509 -req -CA ca.cert -CAkey ca.key -in cert-file -out cert-signed -days 730 -CAcreateserial keytool -keystore kafka.server.keystore.jks -alias CARoot -importcert -file ca.cert -storepass secure_password -noprompt keytool -keystore kafka.server.keystore.jks -alias server01.domain.com -importcert -file cert-signed -storepass secure_password -noprompt
Then the $CONFLUENT_HOME/etc/kafka/kraft/server.properties:
listeners=BROKER://:9092,CONTROLLER://:9093 inter.broker.listener.name=BROKER advertised.listeners=BROKER://server01.domain.com:9092,CONTROLLER://server01.domain.com:9093 controller.listener.names=CONTROLLER listener.security.protocol.map=BROKER:SASL_SSL,CONTROLLER:SASL_SSL sasl.enabled.mechanisms=PLAIN sasl.mechanism.inter.broker.protocol=PLAIN sasl.mechanism.controller.protocol=PLAIN #security.protocol=SASL_PLAINTEXT security.protocol=SASL_SSL listener.name.broker.plain.sasl.jaas.config = org.apache.kafka.common.security.plain.PlainLoginModule required \ username="admin" \ password="admin-secret" \ user_admin="admin-secret" \ user_schemaregistry="schemaregistry-secret" \ user_yjaquier="secure_password"; listener.name.controller.plain.sasl.jaas.config = org.apache.kafka.common.security.plain.PlainLoginModule required \ username="admin" \ password="admin-secret" \ user_admin="admin-secret" \ user_schemaregistry="schemaregistry-secret" \ user_yjaquier="secure_password"; ssl.client.auth=required #ssl.endpoint.identification.algorithm= # Avoid to set: unsecure ssl.truststore.location=/home/kafka/ssl/kafka.server.truststore.jks ssl.truststore.password=secure_password ssl.keystore.location=/home/kafka/ssl/kafka.server.keystore.jks ssl.keystore.password=secure_password
Use five below commands to ensure the CN has been correctly filled for the keystore of the server and the truststore of the server/client:
keytool -list -v -keystore kafka.server.keystore.jks -storepass secure_password keytool -list -v -keystore kafka.server.truststore.jks -storepass secure_password keytool -list -v -keystore kafka.client.truststore.jks -storepass secure_password openssl x509 -in ca.cert -text -noout openssl x509 -in cert-signed -text -noout
Start the Kafka server (broker and controller) with something like:
nohup kafka-server-start $CONFLUENT_HOME/etc/kafka/kraft/server.properties > ~/kafka-server-start.log 2>&1 & clear;kafka-server-start $CONFLUENT_HOME/etc/kafka/kraft/server.properties
Once the Kafka broker is up you can check the CN in its certificate with:
openssl s_client -connect server01.domain.com:9092 -showcerts
My Kafka client configuration file (client.conf) is now:
security.protocol=SASL_SSL
sasl.mechanism=PLAIN
sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required \
username="yjaquier" \
password="secure_password";
ssl.truststore.location=/home/kafka/ssl/kafka.client.truststore.jks
ssl.truststore.password=secure_password
#ssl.endpoint.identification.algorithm= # unsecure
Start a consumer with:
kafka-console-consumer --topic oracle-kafka --bootstrap-server server01.domain.com:9092 --consumer.config ~/client.conf --from-beginning
In my producer Python code I now have:
producer = KafkaProducer(bootstrap_servers="server01.domain.com:9092",
security_protocol='SASL_SSL',
sasl_mechanism='PLAIN',
sasl_plain_username="yjaquier",
sasl_plain_password="secure_password",
value_serializer=lambda m: json.dumps(m).encode('utf-8'), max_block_ms=5000)
You should get:
Error creating Kafka producer: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self-signed certificate in certificate chain (_ssl.c:1028)
To solve this, extract the CA from the client truststore (or server or server keystore as the CA is present in all of them) with:
keytool -exportcert -alias CARoot -keystore kafka.client.truststore.jks -rfc -file CARoot.pem -storepass secure_password
Transfer this CARoot.pem file in the directory of your Python script and change the producer with:
producer = KafkaProducer(bootstrap_servers="server01.domain.com:9092",
security_protocol='SASL_SSL',
sasl_mechanism='PLAIN',
sasl_plain_username="yjaquier",
sasl_plain_password="secure_password",
ssl_cafile='CARoot.pem',
# ssl_check_hostname=False,
value_serializer=lambda m: json.dumps(m).encode('utf-8'), max_block_ms=5000)
And you should get:
$ python kafka_apache.py --type producer Successfully created Kafka producer Message sent successfully.
Kafka TLS/SSL with Subject Alternative Name (SAN)
Even if using CN is valid SAN is recommended as stated in Confluent documentation:
Both fields are valid, however RFC-2818 recommends the use of SAN. SAN is also more flexible, allowing for multiple DNS entries to be declared. Another advantage is that the CN can be set to a more meaningful value for authorization purposes. To add a SAN field, append the argument -ext SAN=DNS:{FQDN} to the keytool command:
The Confluent documentation is not clear at all for SAN configuration and I had to fight a lot. Particularly signing the certificate with the CA and the -extfile and -extensions option of openssl. I had this issue because the controller process was not able to authenticate with broker because the signed certificate did not contain the SAN name of the server. My openssl.conf.san file is also much more complete:
[kafka@server01 ssl]$ cat openssl.conf.san [ req ] default_bits = 2048 encrypt_key = no default_md = sha256 utf8 = yes string_mask = utf8only prompt = no distinguished_name = req_distinguished_name x509_extensions = req_ext [ req_distinguished_name ] countryName = CH stateOrProvinceName = Geneva localityName = Geneva organizationName = MyCompany organizationalUnitName = IT commonName = Yannick Jaquier [ req_ext ] basicConstraints = CA:TRUE keyUsage = digitalSignature, keyCertSign subjectAltName = @alt_names [ alt_names ] DNS.1 = server01.domain.com DNS.2 = server01 DNS.3 = localhost IP.1 = 10.75.43.64
Remark:
I have deliberately chosen a different CN to avoid mis-configuration
Not entering again in details and my generate.sh.san script is (I have also used a CN different from server name):
rm *.jks ca* cert* *.pem keytool -keystore kafka.server.keystore.jks -alias server01.domain.com -keyalg RSA -validity 730 -genkeypair -ext SAN=dns:server01.domain.com,dns:server01,ip:10.75.43.64 -storepass secure_password -dname "CN=Yannick Jaquier, OU=IT, O=IT, L=Geneva, S=Geneva, C=CH" openssl req -new -x509 -keyout ca.key -out ca.cert -days 730 -config openssl.conf.san keytool -keystore kafka.client.truststore.jks -alias CARoot -importcert -file ca.cert -ext SAN=dns:server01.domain.com,dns:server01,ip:10.75.43.64 -storepass secure_password -noprompt keytool -keystore kafka.server.truststore.jks -alias CARoot -importcert -file ca.cert -ext SAN=dns:server01.domain.com,dns:server01,ip:10.75.43.64 -storepass secure_password -noprompt keytool -keystore kafka.server.keystore.jks -alias server01.domain.com -certreq -file cert-file -ext SAN=dns:server01.domain.com,dns:server01,ip:10.75.43.64 -storepass secure_password -noprompt openssl x509 -req -CA ca.cert -CAkey ca.key -in cert-file -out cert-signed -days 730 -CAcreateserial -extfile openssl.conf.san -extensions req_ext keytool -keystore kafka.server.keystore.jks -alias CARoot -importcert -file ca.cert -ext SAN=dns:server01.domain.com,dns:server01,ip:10.75.43.64 -storepass secure_password -noprompt keytool -keystore kafka.server.keystore.jks -alias server01.domain.com -importcert -file cert-signed -ext SAN=dns:server01.domain.com,dns:server01,ip:10.75.43.64 -storepass secure_password -noprompt
In the middle of re-starting your Kafka server ensure all certificate contains a SAN entry with those commands:
keytool -list -v -keystore kafka.server.keystore.jks -storepass secure_password keytool -list -v -keystore kafka.server.truststore.jks -storepass secure_password keytool -list -v -keystore kafka.client.truststore.jks -storepass secure_password openssl x509 -in ca.cert -text -noout openssl x509 -in cert-signed -text -noout
Example:
[kafka@server01 ssl]$ openssl x509 -in cert-signed -text -noout
.
.
X509v3 extensions:
X509v3 Basic Constraints:
CA:TRUE
X509v3 Key Usage:
Digital Signature, Certificate Sign
X509v3 Subject Alternative Name:
DNS:server01.domain.com, DNS:server01, DNS:localhost, IP Address:10.75.43.64
.
.
.
The Python testing remains the same…
Access Control Lists (ACLs)
Once your users need to authenticate, one step forward in security is to give and forbid some users to perform some operations. For example how would you restrict a Kafka topics to be open for everyone for producing ?
For Kraft Kafka server add the following lie in $CONFLUENT_HOME/etc/kafka/kraft/server.properties file:
authorizer.class.name=org.apache.kafka.metadata.authorizer.StandardAuthorizer super.users=User:admin
And restart your Kafka Server…
Now if I try to launch a consumer with my yjaquier account I get:
[kafka@server01 ~]$ kafka-console-consumer --topic oracle-kafka --bootstrap-server server01.domain.com:9092 --consumer.config ~/client.conf --from-beginning
[2025-06-10 15:15:07,746] WARN [Consumer clientId=console-consumer, groupId=console-consumer-18100] The metadata response from the cluster reported a recoverable issue with correlation id 2 : {oracle-kafka=TOPIC_AUTHORIZATION_FAILED} (org.apache.kafka.clients.NetworkClient)
[2025-06-10 15:15:07,752] ERROR [Consumer clientId=console-consumer, groupId=console-consumer-18100] Topic authorization failed for topics [oracle-kafka] (org.apache.kafka.clients.Metadata)
[2025-06-10 15:15:07,756] ERROR Error processing message, terminating consumer process: (org.apache.kafka.tools.consumer.ConsoleConsumer)
org.apache.kafka.common.errors.TopicAuthorizationException: Not authorized to access topics: [oracle-kafka]
Processed a total of 0 messages
Same with Python producer client:
$ python kafka_apache.py --type producer Successfully created Kafka producer ERROR:kafka.cluster:Topic oracle-kafka is not authorized for this client ERROR:kafka.producer.kafka:kafkaproducer client_id="kafka-python-producer-1" transactional_id="None": Exception occurred during message send: [Error 29] TopicAuthorizationFailedError: oracle-kafka Message sent successfully.
Remark:
The misisng authorization is strangely not generating an exception…
I have built this adminclient.conf configuration file:
[kafka@server01 ~]$ cat adminclient.conf
security.protocol=SASL_SSL
sasl.mechanism=PLAIN
sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required \
username="admin" \
password="admin-secret";
ssl.truststore.location=/home/kafka/ssl/kafka.client.truststore.jks
ssl.truststore.password=secure_password
Initially I have no grants on my Kafka topic oracle-kafka:
[kafka@server01 ~]$ kafka-acls --bootstrap-server server01.domain.com:9092 --command-config ~/adminclient.conf --list --topic oracle-kafka
I add my yjaquier account to be able to read from the topic from any host (I do not specify the group option):
[kafka@server01 ~]$ kafka-acls --bootstrap-server server01.domain.com:9092 --command-config ~/adminclient.conf \
> --add \
> --allow-principal User:yjaquier \
> --operation read \
> --topic oracle-kafka
Adding ACLs for resource `ResourcePattern(resourceType=TOPIC, name=oracle-kafka, patternType=LITERAL)`:
(principal=User:yjaquier, host=*, operation=READ, permissionType=ALLOW)
[kafka@server01~]$ kafka-acls --bootstrap-server server01.domain.com:9092 --command-config ~/adminclient.conf --list --topic oracle-kafka
Current ACLs for resource `ResourcePattern(resourceType=TOPIC, name=oracle-kafka, patternType=LITERAL)`:
(principal=User:yjaquier, host=*, operation=READ, permissionType=ALLOW)
I got this error. The group name is dynamically generated to impossible to guess it in advance. I could have specified it but we have already seen that it memorized the status of what we have already read but clearly this is an option to consider for extra security:
[kafka@server01 ~]$ kafka-console-consumer --topic oracle-kafka --bootstrap-server server01.domain.com:9092 --consumer.config ~/client.conf --from-beginning [2025-06-10 15:43:13,260] ERROR Error processing message, terminating consumer process: (org.apache.kafka.tools.consumer.ConsoleConsumer) org.apache.kafka.common.errors.GroupAuthorizationException: Not authorized to access group: console-consumer-6192 Processed a total of 0 messages
So granting for any group name:
[kafka@server01 ~]$ kafka-acls --bootstrap-server server01.domain.com:9092 --command-config ~/adminclient.conf \
> --add \
> --allow-principal User:yjaquier \
> --operation read \
> --topic oracle-kafka \
> --group '*'
Adding ACLs for resource `ResourcePattern(resourceType=TOPIC, name=oracle-kafka, patternType=LITERAL)`:
(principal=User:yjaquier, host=*, operation=READ, permissionType=ALLOW)
Adding ACLs for resource `ResourcePattern(resourceType=GROUP, name=*, patternType=LITERAL)`:
(principal=User:yjaquier, host=*, operation=READ, permissionType=ALLOW)
[kafka@server01 ~]$ kafka-acls --bootstrap-server server01.domain.com:9092 --command-config ~/adminclient.conf --list --topic oracle-kafka
Current ACLs for resource `ResourcePattern(resourceType=TOPIC, name=oracle-kafka, patternType=LITERAL)`:
(principal=User:yjaquier, host=*, operation=READ, permissionType=ALLOW)
After this I’m able to launch a consumer process directly from the server but still it is not possible to produce a message:
[kafka@server01 ~]$ kafka-console-producer --topic oracle-kafka --bootstrap-server server01.domain.com:9092 --producer.config ~/client.conf
>{"key": "Random value 1234"}
[2025-06-10 16:26:20,996] ERROR [Producer clientId=console-producer] Aborting producer batches due to fatal error (org.apache.kafka.clients.producer.internals.Sender)
org.apache.kafka.common.errors.ClusterAuthorizationException: Cluster authorization failed.
[2025-06-10 16:26:21,006] ERROR Error when sending message to topic oracle-kafka with key: null, value: 28 bytes with error: (org.apache.kafka.clients.producer.internals.ErrorLoggingCallback)
org.apache.kafka.common.errors.ClusterAuthorizationException: Cluster authorization failed.
org.apache.kafka.common.KafkaException: Cannot execute transactional method because we are in an error state
at org.apache.kafka.clients.producer.internals.TransactionManager.maybeFailWithError(TransactionManager.java:1042)
at org.apache.kafka.clients.producer.internals.TransactionManager.maybeAddPartition(TransactionManager.java:386)
at org.apache.kafka.clients.producer.KafkaProducer.doSend(KafkaProducer.java:1092)
at org.apache.kafka.clients.producer.KafkaProducer.send(KafkaProducer.java:993)
at kafka.tools.ConsoleProducer$.send(ConsoleProducer.scala:118)
at kafka.tools.ConsoleProducer$.loopReader(ConsoleProducer.scala:91)
at kafka.tools.ConsoleProducer$.main(ConsoleProducer.scala:100)
at kafka.tools.ConsoleProducer.main(ConsoleProducer.scala)
Caused by: org.apache.kafka.common.errors.ClusterAuthorizationException: Cluster authorization failed.
From Python no error is produced but inside $CONFLUENT_HOME/logs/kafka-authorizer.log I found:
[2025-06-10 16:36:17,030] INFO Principal = User:yjaquier is Denied operation = WRITE from host = 10.71.17.6 on resource = Topic:LITERAL:oracle-kafka for request = Produce with resourceRefCount = 1 based on rule DefaultDeny (kafka.authorizer.logger)
I authorize my account to write on my oracle-kafka topic:
[kafka@server01 ~]$ kafka-acls --bootstrap-server server01.domain.com:9092 --command-config ~/adminclient.conf \
> --add \
> --allow-principal User:yjaquier \
> --operation write \
> --topic oracle-kafka
Adding ACLs for resource `ResourcePattern(resourceType=TOPIC, name=oracle-kafka, patternType=LITERAL)`:
(principal=User:yjaquier, host=*, operation=WRITE, permissionType=ALLOW)
[kafka@server01 ~]$ kafka-acls --bootstrap-server server01.domain.com:9092 --command-config ~/adminclient.conf --list --topic oracle-kafka
Current ACLs for resource `ResourcePattern(resourceType=TOPIC, name=oracle-kafka, patternType=LITERAL)`:
(principal=User:yjaquier, host=*, operation=WRITE, permissionType=ALLOW)
(principal=User:yjaquier, host=*, operation=READ, permissionType=ALLOW)
And now producing and consuming messages is possible using my account…
Issues encountered
org.apache.kafka.common.errors.SslAuthenticationException: SSL handshake failed
When trying to start Kafka server in SAN mode I got:
[2025-05-28 17:20:10,929] ERROR [NodeToControllerChannelManager id=1 name=heartbeat] Connection to node 1 (server01.domain.com/10.75.43.64:9093) failed authentication due to: SSL handshake failed (org.apache.kafka.clients.NetworkClient)
[2025-05-28 17:20:10,930] ERROR [broker-1-to-controller-heartbeat-channel-manager]: Failed to send the following request due to authentication error: ClientRequest(expectResponse=true, callback=kafka.server.NodeToControllerRequestThread$$Lambda$807/0x00007f194c4110d8@d20b551, destination=1, correlationId=3, clientId=1, createdTimeMs=1748445610899, requestBuilder=BrokerRegistrationRequestData(brokerId=1, clusterId='PYNUg5Q3TnqyqmT8FzEsIg', incarnationId=ISQb59yUQWiuTM7jSx25LA, listeners=[Listener(name='BROKER', host='server01.domain.com', port=9092, securityProtocol=3)], features=[Feature(name='kraft.version', minSupportedVersion=0, maxSupportedVersion=1), Feature(name='metadata.version', minSupportedVersion=1, maxSupportedVersion=21)], rack=null, isMigratingZkBroker=false, logDirs=[j3LJIuLzech7S5qEFbckyA], previousBrokerEpoch=-1)) (kafka.server.NodeToControllerRequestThread)
[2025-05-28 17:20:10,930] INFO [SocketServer listenerType=CONTROLLER, nodeId=1] Failed authentication with /10.75.43.64 (channelId=10.75.43.64:9093-10.75.43.64:46476-2) (SSL handshake failed) (org.apache.kafka.common.network.Selector)
[2025-05-28 17:20:10,930] ERROR [broker-1-to-controller-heartbeat-channel-manager]: Request BrokerRegistrationRequestData(brokerId=1, clusterId='PYNUg5Q3TnqyqmT8FzEsIg', incarnationId=ISQb59yUQWiuTM7jSx25LA, listeners=[Listener(name='BROKER', host='server01.domain.com', port=9092, securityProtocol=3)], features=[Feature(name='kraft.version', minSupportedVersion=0, maxSupportedVersion=1), Feature(name='metadata.version', minSupportedVersion=1, maxSupportedVersion=21)], rack=null, isMigratingZkBroker=false, logDirs=[j3LJIuLzech7S5qEFbckyA], previousBrokerEpoch=-1) failed due to authentication error with controller. Disconnecting the connection to the stale controller 1 (kafka.server.NodeToControllerRequestThread)
org.apache.kafka.common.errors.SslAuthenticationException: SSL handshake failed
Caused by: javax.net.ssl.SSLHandshakeException: No name matching server01.domain.com found
at java.base/sun.security.ssl.Alert.createSSLException(Alert.java:131)
at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:383)
at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:326)
at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:321)
at java.base/sun.security.ssl.CertificateMessage$T13CertificateConsumer.checkServerCerts(CertificateMessage.java:1351)
at java.base/sun.security.ssl.CertificateMessage$T13CertificateConsumer.onConsumeCertificate(CertificateMessage.java:1226)
at java.base/sun.security.ssl.CertificateMessage$T13CertificateConsumer.consume(CertificateMessage.java:1169)
at java.base/sun.security.ssl.SSLHandshake.consume(SSLHandshake.java:396)
at java.base/sun.security.ssl.HandshakeContext.dispatch(HandshakeContext.java:480)
at java.base/sun.security.ssl.SSLEngineImpl$DelegatedTask$DelegatedAction.run(SSLEngineImpl.java:1277)
at java.base/sun.security.ssl.SSLEngineImpl$DelegatedTask$DelegatedAction.run(SSLEngineImpl.java:1264)
at java.base/java.security.AccessController.doPrivileged(AccessController.java:712)
at java.base/sun.security.ssl.SSLEngineImpl$DelegatedTask.run(SSLEngineImpl.java:1209)
at org.apache.kafka.common.network.SslTransportLayer.runDelegatedTasks(SslTransportLayer.java:444)
at org.apache.kafka.common.network.SslTransportLayer.handshakeUnwrap(SslTransportLayer.java:533)
at org.apache.kafka.common.network.SslTransportLayer.doHandshake(SslTransportLayer.java:382)
at org.apache.kafka.common.network.SslTransportLayer.handshake(SslTransportLayer.java:302)
at org.apache.kafka.common.network.KafkaChannel.prepare(KafkaChannel.java:178)
at org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:563)
at org.apache.kafka.common.network.Selector.poll(Selector.java:501)
at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:596)
at org.apache.kafka.server.util.InterBrokerSendThread.pollOnce(InterBrokerSendThread.java:110)
at kafka.server.NodeToControllerRequestThread.doWork(NodeToControllerChannelManager.scala:377)
at org.apache.kafka.server.util.ShutdownableThread.run(ShutdownableThread.java:136)
Caused by: java.security.cert.CertificateException: No name matching server01.domain.com found
at java.base/sun.security.util.HostnameChecker.matchDNS(HostnameChecker.java:234)
at java.base/sun.security.util.HostnameChecker.match(HostnameChecker.java:103)
at java.base/sun.security.ssl.X509TrustManagerImpl.checkIdentity(X509TrustManagerImpl.java:467)
at java.base/sun.security.ssl.X509TrustManagerImpl.checkIdentity(X509TrustManagerImpl.java:418)
at java.base/sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:292)
at java.base/sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:144)
at java.base/sun.security.ssl.CertificateMessage$T13CertificateConsumer.checkServerCerts(CertificateMessage.java:1329)
... 19 more
This was because the singed certificate did not contains the SAN name of the server. The Confluent documentation is incomplete so refer to my SAN creation script above…
Error creating Kafka producer: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: Hostname mismatch, certificate is not valid for ‘server01.domain.com’. (_ssl.c:1028)
For testing purpose you can set this parameter in your producer/consumer Python script:
ssl_check_hostname=False
But the source error is most probably that the server name has not correctly been added in your certificates (you have also surely use the server counter part parameter called ssl.endpoint.identification.algorithm).
Error creating Kafka producer: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self-signed certificate in certificate chain (_ssl.c:1028)
You must export the root certificate with something like:
[kafka@server01 ssl]$ keytool -exportcert -alias CARoot -keystore kafka.client.truststore.jks -rfc -file CARoot.pem -storepass secure_password Certificate stored in file caroot.pem
Then transfer this file to your client and use it in your script (Python or else):
ssl_cafile='CARoot.pem'
The complete error message is:
[2025-06-10 12:40:37,973] ERROR [ControllerApis nodeId=1] Unexpected error handling request RequestHeader(apiKey=CONTROLLER_REGISTRATION, apiVersion=0, clientId=1, correlationId=0, headerVersion=2) -- ControllerRegistrationRequestData(controllerId=1, incarnationId=hJ14_M8MRrWQp2g-yMFuiA, zkMigrationReady=false, listeners=[Listener(name='CONTROLLER', host='server01.domain.com', port=9093, securityProtocol=3)], features=[Feature(name='kraft.version', minSupportedVersion=0, maxSupportedVersion=1), Feature(name='metadata.version', minSupportedVersion=1, maxSupportedVersion=21)]) with context RequestContext(header=RequestHeader(apiKey=CONTROLLER_REGISTRATION, apiVersion=0, clientId=1, correlationId=0, headerVersion=2), connectionId='10.75.43.64:9093-10.75.43.64:38898-0', clientAddress=/10.75.43.64, principal=User:admin, listenerName=ListenerName(CONTROLLER), securityProtocol=SASL_SSL, clientInformation=ClientInformation(softwareName=apache-kafka-java, softwareVersion=7.9.1-ccs), fromPrivilegedListener=false, principalSerde=Optional[org.apache.kafka.common.security.authenticator.DefaultKafkaPrincipalBuilder@1206bdbd]) (kafka.server.ControllerApis) org.apache.kafka.common.errors.ClusterAuthorizationException: Request Request(processor=0, connectionId=10.75.43.64:9093-10.75.43.64:38898-0, session=org.apache.kafka.network.Session@5c96d4bf, listenerName=ListenerName(CONTROLLER), securityProtocol=SASL_SSL, buffer=null, envelope=None) is not authorized. [2025-06-10 12:40:37,979] INFO [ControllerRegistrationManager id=1 incarnation=hJ14_M8MRrWQp2g-yMFuiA] RegistrationResponseHandler: controller returned error CLUSTER_AUTHORIZATION_FAILED (Cluster authorization failed.) (kafka.server.ControllerRegistrationManager)
And found in authorizer log:
[kafka@server01 ~]$ tail $CONFLUENT_HOME/logs/kafka-authorizer.log . [2025-06-10 12:40:37,970] INFO Principal = User:admin is Denied operation = CLUSTER_ACTION from host = 10.75.43.64 on resource = Cluster:LITERAL:kafka-cluster for request = ControllerRegistration with resourceRefCount = 1 based on rule DefaultDeny (kafka.authorizer.logger) .
You need at least one super account to perform the BROKER/CONTROLLER stuff so add something like in your configuration:
super.users=User:admin
References
- Enable Security for a ZooKeeper-Based Cluster in Confluent Platform
- Protect Data in Motion with TLS Encryption in Confluent Platform
- Use Access Control Lists (ACLs) for Authorization in Confluent Platform
- Kafka Authentication with SSL and SASL_SSL
- Mermaid Diagramming and charting tool





