Saga pattern: transactions across microservices
Placing an order touches payment, stock and shipping in different services. A saga keeps them consistent without a distributed transaction.
In a monolith with one database, placing an order is one transaction: reserve stock, charge payment, create shipment, and if anything fails, everything rolls back. With microservices, each service has its own database, so no single transaction spans them. A saga is a sequence of local transactions, with a compensating action to undo each step if a later one fails.
Example: placing an order
- Order service: create order as
PENDING. - Inventory service: reserve items.
- Payment service: charge the customer.
- Shipping service: schedule delivery.
- Order service: mark
CONFIRMED.
If payment fails at step 3, the saga runs compensations in reverse: release the reserved items, then cancel the order.
Compensation is not rollback
A compensating action is a new business action, such as "refund the payment" or "cancel the shipment". It may not restore the world exactly (an email may already be sent), so design steps so that compensations make business sense.
Two ways to coordinate
Choreography. Each service listens for events and reacts, publishing its own event. No central controller.
- Simple for a few steps, loosely coupled.
- Hard to see the whole flow and to debug as it grows.
Orchestration. A central orchestrator tells each service what to do next and handles failures.
- Clear flow, easier to monitor and change.
- The orchestrator is another component to build and keep reliable.
What you must handle
- Idempotency: messages can repeat, so every step and compensation must be safe to run twice.
- Failure of the compensation itself: retry until it succeeds, and alert a human if it cannot.
- Visibility into half-done states: other users may briefly see an order that is pending. Show the state honestly.
- Persist saga state so the process survives a crash.
- Timeouts for steps that never answer.
When to avoid it
If you can keep related data in one service and one database, do that. A saga is the price of splitting data across services.
Takeaway: use sagas for cross-service workflows. Prefer orchestration as the flow grows, and make every step and its compensation idempotent.
System Design series, Part 2 (building blocks): topic 14 of 14. Previous: Event sourcing: store events, not just the current state.
