Message queues: why systems use them and how they fail

A message queue sits between a producer (which sends work) and a consumer (which does it). The producer drops a message in and moves on. The consumer takes it when ready.

Why use one

• Decoupling: services do not need to be up at the same moment.
• Absorb spikes: a traffic surge piles up in the queue instead of crashing the worker.
• Async work: send the email, resize the image or generate the report in the background, so the user's request returns fast.
• Retry and scale: add more consumers to drain the queue faster.

Delivery guarantees

• At-most-once: may lose messages, never duplicates.
• At-least-once: never loses, but may deliver twice. This is the common choice, so consumers must be idempotent (see that part).
• Exactly-once: very hard in general; usually it means at-least-once plus deduplication.

When processing fails

Retry a few times with a growing delay. If it still fails, move the message to a dead-letter queue (DLQ) so it does not block others, and alert someone to inspect it. A "poison message" that always fails is the classic reason a consumer gets stuck.

Ordering

A single queue can keep order, but many parallel consumers break it. Systems like Kafka keep order within a partition, so you pick a key (such as user_id) that sends related messages to the same partition.

Choosing a tool

• RabbitMQ / SQS: task queues; a message is consumed and gone. Flexible routing.
• Kafka: a durable log; messages stay for a set time and many consumers can replay them. Great for event streams and high volume.

Watch out

Queues grow silently. Monitor queue depth and age of the oldest message.

Takeaway: use a queue for anything slow or spiky, expect duplicates, and always have a dead-letter path.

---

Part 12 of 15 in the System Design series. Previous: Rate limiting: token bucket, sliding window and how to build it.