At-Least-Once Delivery & Idempotent Consumers

Foundations

In the first lesson, a lost or repeated message cost only a slightly wrong count. Now, what if each message triggers a side effect that must happen once, but the queue may deliver a message twice?

Most queues deliver at least once: a worker that crashes after acting but before acknowledging gets the message again. So consumers must be idempotent, safe to run twice. Give each message a stable id and carry it into the side effect: record the id in the same transaction as a database change, or pass it as the idempotency key of an external call. Retry temporary failures with backoff, and move a message that keeps failing to a dead-letter queue.

Take message m1, whose effect succeeds but whose acknowledgement is lost. The queue redelivers m1; the consumer finds m1 already recorded and skips it. Message m2 fails three times on a permanently bad input, so it moves to the dead-letter queue, and m3 proceeds instead of waiting behind it.

• Queues

  Work moves off the request path into a queue.

  Open lesson

Case Scenario

To see how idempotent consumers make redelivery harmless, consider push notifications after a post on Instagram: users mentioned in the post get a push, and the push provider sometimes fails a request that succeeds on retry. Each notification carries a stable id into the provider call, so a redelivered message never sends a second push.

A queue can redeliver m1 after a worker finishes but loses its acknowledgement, and a push provider can time out after actually notifying Ana. Another delivery may keep failing because its device token is invalid. Duplicate suppression needs a stable id all the way to the side effect, and one poison message must never stall the queue.