Async Work Off the Request Path

Foundations

A message queue sits between a producer that publishes small messages and a consumer, or worker, that processes them later at its own pace. Moving work into a queue takes it off the request path: the request publishes a message and responds at once, and the slow work finishes afterwards. A burst waits in the queue instead of slowing every caller.

Asynchronous work fits when the caller does not need the result in its response, and a delay, or an occasional loss, is acceptable for it. Workers can batch messages, combining many updates into one write. The cost is eventual consistency: whatever the worker updates lags behind, by as much as the queue's backlog.

Take a service that records a metric on every request, 1,000 requests per second. Writing it inline adds a database write to each response. Publishing an event instead costs a fast enqueue; a worker reads batches of 500 events and writes each batch's totals in one update, about 2 writes per second instead of 1,000.

• Databases

  Queues and partitions sit around a primary database. That lesson covers the key and the index.

  Open lesson

Case Scenario

To see how a queue takes work off the request path, consider click counting in a URL shortener like bit.ly: every redirect may be counted for the link's owner, but the visitor's 301 must not wait for a counter to commit. The redirect publishes a click event, and a worker batches the counts into a few writes.

A click count is useful to the owner but irrelevant to the visitor waiting for a 301. Updating the same counter during every open adds a write to the hottest path, and a viral x7 can concentrate those writes on one row. Losing a small fraction of counts is acceptable; losing or delaying the redirect is not, so the 301 must not wait for a counter commit.