Home/Learn/Apache Kafka/Kafka Security (TLS & SASL)

Kafka Security (TLS & SASL)

Advanced

Administration

Enable TLS for encryption in transit; use SASL/PLAIN, SASL/SCRAM, or SASL/OAUTHBEARER for authentication; ACLs control per-topic produce/consume permissions.

Overview

Kafka security has three pillars: encryption (TLS/SSL for data in transit), authentication (who is connecting), and authorisation (what they can do). TLS encrypts all broker-to-client and inter-broker communication using certificates. Authentication can be SASL/PLAIN (username/password — only over TLS), SASL/SCRAM (challenge-response with credentials stored in ZooKeeper/KRaft), SASL/GSSAPI (Kerberos for enterprise environments), or SASL/OAUTHBEARER (JWT tokens from an OAuth2 provider). Authorisation uses ACLs (Access Control Lists) or a custom Authorizer implementation. Spring Boot's spring-kafka supports all authentication mechanisms through producer/consumer property injection.

TLS encryption configuration

TLS encrypts data between clients and brokers. Generate a CA, broker keystore (broker certificate), and client truststore (CA certificate that validates broker certs). One-way TLS (clients verify broker) is most common; mutual TLS (mTLS) also requires clients to present certificates. Configure brokers with listeners using SSL protocol.

Config + YAML — broker TLS and Spring Boot client SSL configuration

# server.properties — broker TLS configuration
listeners=PLAINTEXT://:9092,SSL://:9093
advertised.listeners=PLAINTEXT://broker1:9092,SSL://broker1:9093
listener.security.protocol.map=PLAINTEXT:PLAINTEXT,SSL:SSL

ssl.keystore.location=/etc/kafka/ssl/broker.keystore.jks
ssl.keystore.password=keystore-password
ssl.key.password=key-password
ssl.truststore.location=/etc/kafka/ssl/broker.truststore.jks
ssl.truststore.password=truststore-password
ssl.client.auth=none          # none=one-way TLS, required=mTLS

# application.yml — Spring Boot client TLS config
spring:
  kafka:
    properties:
      security.protocol: SSL
      ssl.truststore.location: classpath:ssl/client.truststore.jks
      ssl.truststore.password: ${KAFKA_TRUSTSTORE_PASSWORD}
      # For mTLS (client certificate)
      ssl.keystore.location: classpath:ssl/client.keystore.jks
      ssl.keystore.password: ${KAFKA_KEYSTORE_PASSWORD}
      ssl.key.password: ${KAFKA_KEY_PASSWORD}

SASL authentication — PLAIN and SCRAM

SASL/PLAIN sends credentials as plaintext — only use over TLS. SASL/SCRAM-SHA-256/512 stores credentials as salted hashes in ZooKeeper/KRaft and uses challenge-response — safer than PLAIN. SASL/OAUTHBEARER delegates authentication to an OAuth2 provider using JWT tokens. Always combine SASL with TLS (SASL_SSL protocol).

Config + YAML — SASL/SCRAM broker and Spring Boot client configuration

# server.properties — SASL/SCRAM configuration
listeners=SASL_SSL://:9093
advertised.listeners=SASL_SSL://broker1:9093
listener.security.protocol.map=SASL_SSL:SASL_SSL
sasl.enabled.mechanisms=SCRAM-SHA-256
sasl.mechanism.inter.broker.protocol=SCRAM-SHA-256

# Create user credentials in Kafka (KRaft / ZooKeeper)
kafka-configs.sh --bootstrap-server broker:9093 \
  --alter --entity-type users --entity-name order-service \
  --add-config 'SCRAM-SHA-256=[password=secret123]'

# application.yml — Spring Boot SASL/SCRAM client
spring:
  kafka:
    properties:
      security.protocol: SASL_SSL
      sasl.mechanism: SCRAM-SHA-256
      sasl.jaas.config: >
        org.apache.kafka.common.security.scram.ScramLoginModule required
        username="order-service"
        password="${KAFKA_PASSWORD}";
      ssl.truststore.location: classpath:ssl/client.truststore.jks
      ssl.truststore.password: ${KAFKA_TRUSTSTORE_PASSWORD}

ACLs for authorisation

ACLs grant/deny produce, consume, and describe permissions per principal (user/service account) for specific topics, consumer groups, or cluster operations. With KRaft, ACLs are managed via kafka-acls.sh or the Admin API. Authorisation is enabled by setting authorizer.class.name in server.properties. The AclAuthorizer allows listed super.users to bypass all ACL checks.

CLI — ACL grant, list, and revoke for topic and consumer group access

# server.properties — enable ACL authoriser (KRaft)
authorizer.class.name=org.apache.kafka.metadata.authorizer.StandardAuthorizer
super.users=User:admin;User:kafka-broker   # bypass all ACLs

# Grant order-service WRITE access to the orders topic
kafka-acls.sh --bootstrap-server broker:9093 \
  --add \
  --allow-principal User:order-service \
  --operation Write \
  --topic orders

# Grant analytics-service READ access to orders + consumer group access
kafka-acls.sh --bootstrap-server broker:9093 \
  --add \
  --allow-principal User:analytics-service \
  --operation Read \
  --topic orders

kafka-acls.sh --bootstrap-server broker:9093 \
  --add \
  --allow-principal User:analytics-service \
  --operation Read \
  --group analytics-consumer-group

# List ACLs for a topic
kafka-acls.sh --bootstrap-server broker:9093 \
  --list --topic orders

# Remove an ACL
kafka-acls.sh --bootstrap-server broker:9093 \
  --remove \
  --allow-principal User:order-service \
  --operation Write \
  --topic orders

Key Points to Remember

1Kafka security = TLS (encryption) + SASL (authentication) + ACLs (authorisation) — all three work independently or together
2SASL/PLAIN transmits credentials in plaintext — always pair with TLS (use SASL_SSL protocol, not SASL_PLAINTEXT)
3SASL/SCRAM-SHA-256/512 stores salted credential hashes — safer than PLAIN; use for non-Kerberos environments
4Consumer groups also need ACLs — READ permission on the consumer group is required alongside READ on the topic
5super.users in server.properties bypass all ACL checks — restrict this list to broker inter-communication only
6SASL/OAUTHBEARER + JWT allows integration with an existing OAuth2 identity provider (Keycloak, Okta)

Interview Questions

What is the difference between SASL/PLAIN and SASL/SCRAM in Kafka, and when would you use each?

MediumConfluent

Why must a consumer have ACL permissions on both the topic and the consumer group?

MediumAmazon

What does SASL_SSL mean as a security.protocol and how does it combine authentication and encryption?

EasyThoughtworks

How would you rotate broker TLS certificates without downtime in a production Kafka cluster?

HardNetflix

How would you implement service-to-service Kafka authentication using SASL/OAUTHBEARER with Keycloak?

HardUber

Ask Aria about Kafka Security (TLS & SASL)

Your personal AI tutor — ask anything about this concept

Revision Status

Personal Notes

Discussion

Loading discussion…

Topic Configuration & Tuning

Kafka Monitoring (JMX/Prometheus)