Idempotency: making retries safe
If a payment request times out, did it go through? Idempotency makes retrying safe. Learn idempotent methods and how idempotency keys work.
An operation is idempotent if doing it many times has the same result as doing it once. Setting a value to 5 is idempotent. Adding 5 to a value is not.
Why it matters
Networks fail. A client sends "charge 100 taka", the connection drops, and it never learns whether the server processed it. If it retries and the operation is not idempotent, the customer may be charged twice. Message queues that deliver at-least-once cause the same issue.
HTTP methods
GET, PUT and DELETE are meant to be idempotent. POST is not by default, and that is where you need extra care.
Idempotency keys
The client generates a unique key (for example a UUID) for each logical operation and sends it in a header such as Idempotency-Key. The server:
- Looks up the key.
- If it is new, processes the request and stores the key together with the response.
- If it already exists, returns the stored response without doing the work again.
Stripe's API popularised this pattern.
Implementation tips
- Store the key with a unique constraint in the database, so two simultaneous retries cannot both win.
- Save the key and the business change in one transaction.
- Mark keys as "in progress" so a parallel duplicate waits or gets a conflict response.
- Keep keys for a limited time (for example 24 hours).
- If the same key arrives with a different body, reject it.
Natural idempotency
Where possible, design operations so they are idempotent anyway: "set status to shipped" instead of "advance status", or use a unique order ID so inserting twice fails the second time.
Takeaway: assume every request can arrive twice, and give every state-changing call an idempotency key.
Part 14 of 15 in the System Design series. Previous: API gateway: the front door of a microservice system.
