Single Message Transforms (SMT)
IntermediateSMTs are lightweight, stateless transformations applied to each message in a Kafka Connect pipeline — routing, masking PII, renaming topics, or flattening nested structures without custom code.
Overview
SMTs run inside the Connect worker process. They are designed for simple, per-message transformations. For complex logic (joins, aggregations), use Kafka Streams or ksqlDB instead.
Common SMT Examples
MaskField, InsertField, RegexRouter, and ExtractField are the most commonly used SMTs.
{
"transforms": "maskPII,addTimestamp,routeByTable",
"transforms.maskPII.type": "org.apache.kafka.connect.transforms.MaskField$Value",
"transforms.maskPII.fields": "email,phone_number",
"transforms.maskPII.replacement": "****",
"transforms.addTimestamp.type": "org.apache.kafka.connect.transforms.InsertField$Value",
"transforms.addTimestamp.timestamp.field": "ingested_at",
"transforms.routeByTable.type": "org.apache.kafka.connect.transforms.RegexRouter",
"transforms.routeByTable.regex": "myapp\.public\.(.*)",
"transforms.routeByTable.replacement": "users-$1"
}Key Points to Remember
- 1SMTs run in-process inside the Connect worker — zero network overhead
- 2SMTs are stateless — cannot join or aggregate across messages
- 3Chain multiple SMTs with a comma-separated "transforms" list
- 4$Key and $Value suffixes select key or value transformation
- 5For stateful logic, use Kafka Streams or ksqlDB, not SMT chains
Interview Questions
Sign in to ask AriaWhat are SMTs in Kafka Connect and what can they do?
How would you mask a PII field using an SMT?
What is the difference between SMTs and Kafka Streams for transformation?
How do you route messages from a Debezium connector to different topics per table?
Can SMTs perform stateful operations like deduplication? Why or why not?
Ask Aria about Single Message Transforms (SMT)
Your personal AI tutor — ask anything about this concept
Revision Status
Personal Notes
Sign in to save personal notes for this topic.
Discussion
Sign in to join the discussion.