· published at SoftwareMill
Reliable Message Delivery: the Transactional Outbox Pattern With Okapi
How the transactional outbox pattern gives you at-least-once delivery without distributed transactions — and how okapi implements it for Kotlin.
Writing to your database and publishing a message to Kafka are two separate operations, and no amount of careful ordering makes them atomic. Commit first and the broker call can fail; publish first and the transaction can roll back. Either way state and events drift apart.
The transactional outbox pattern removes the dual write: the message is inserted into an outbox table inside the same transaction as the business change, and a separate worker drains that table to the broker. One commit, one source of truth, at-least-once delivery without distributed transactions.
This piece walks through the pattern and through okapi — the Kotlin library I maintain under the SoftwareMill OSS umbrella — covering concurrent claiming so two workers never drain the same row, retry policies with retriable/permanent error classification, and the PostgreSQL/MySQL stores plus Kafka and HTTP transports.