What is the outbox pattern, and what problem does it solve?
Quick Answer
Writing to your database and publishing a message are two separate systems — if the process crashes between them, you either lose the event or send a duplicate. The outbox pattern writes the event into an “outbox” table in the same database transaction as the business change, then a separate poller/CDC process reliably publishes it to the message bus and marks it sent, giving atomic write + guaranteed (at-least-once) delivery.
The problem is that saving to your database and publishing a message to a broker are two different systems, so there's no way to make both happen atomically without something like a distributed transaction, which most message brokers don't support anyway. If you save first and the publish fails, you've silently lost the event; if you publish first and the save fails, you've announced something that didn't happen. The outbox pattern fixes it by writing the event as a row in an outbox table, in the exact same local transaction as the business change — so it's atomic with the actual data write, that's just one database. Then a separate process — a poller, or CDC reading the table's change log — picks up unsent rows and publishes them, marking them sent once it succeeds. Because that publish can still be retried, consumers need to be idempotent, deduplicating on the event id.
- The core problem it solves is the “dual write” problem: two independent systems (DB, broker) can't be updated atomically without a distributed transaction.
- Consumers must be idempotent since delivery is at-least-once — dedupe by the event's id.
- The poller can be a simple background job, or CDC (Debezium) reading the outbox table's own change log for lower latency.
- Clean up delivered rows on a schedule so the table doesn't grow forever.