Backpressure & Queue Depth

Foundations

So far, the queue was assumed to keep up. Now, what if messages sometimes arrive faster than workers can finish them, and the backlog must stay within a freshness promise?

Queue depth is the backlog: each second it grows by arrivals and shrinks by what workers finish, never below zero. Depth divided by worker throughput is lag, the seconds a new message waits. Short bursts are fine, since spare capacity drains them. A sustained excess grows depth without limit, so lag past the promise must trigger more workers, an alert, or backpressure that slows producers.

Take workers that finish 1,000 messages per second and arrivals of 800, 1,200, and 1,500 in three seconds. Depth goes 0, 200, then 700; at 1,000 per second, 700 messages is 0.7 seconds of lag. If 1,200 per second kept arriving, depth would grow by 200 every second, and only more workers could catch up.

• Queues

  Work moves off the request path into a queue.

  Open lesson

Case Scenario

To see how queue depth measures lag, consider post fanout in a news feed like X's: a normal user posts, followers' home timelines should include it within a few seconds, and the post call returns as soon as the post is durable. Workers drain the fanout queue, and its depth says whether the freshness promise holds.

A normal author can have hundreds of followers, and each home timeline needs the new post within a few seconds. If the post request waits for every timeline update, its latency grows with follower count and bursts make authors wait for overloaded workers. The post API must return once the post is durable, regardless of follower count, while timeline lag stays within the freshness promise.