Outbox pattern: reliably coupling a database and messages

In short: With the outbox pattern you write the message to be sent in the same database transaction as the change. A separate process reads the outbox table and sends it. That way a change is never lost without its message (or the reverse).

The problem: two systems, one truth

An order should be saved in the database and sent as an event to a message queue. Two separate steps can fail halfway:

• Database written, message not sent (crash in between): nobody learns about the order.
• Message sent, saving fails: the event describes something that doesn’t exist.

A real shared transaction across both systems is expensive (Distributed transactions: two-phase commit and saga).

The solution

1. In one transaction: write a row in orders and a row in outbox (event as text/JSON).
2. A relay reads new outbox rows (polling at short intervals or via the database’s change log) and sends them.
3. After a successful send the row is marked done or deleted.

Important: duplicate messages

If the relay crashes after sending but before marking, the message goes out a second time. The pattern therefore guarantees at least once. Receivers must be idempotent, for example with an event ID that lets them recognise messages they already processed.

When it fits

Whenever a change must reliably trigger an event, for example with Microservices or event sourcing and CQRS. For a single application with one database, a simple table as a queue is often enough.