Stampede Protection (Request Coalescing)

Foundations

So far, each miss reloaded its own value. Now, what if a hot key expires while thousands of requests are arriving, so they all miss at once and all query the store for the same value?

That burst is a cache stampede, or thundering herd. Coalescing fixes it: the first miss takes a short-lived lock for the key and loads the value; later misses see the lock and wait for that result, or serve a slightly stale copy. The lock needs its own expiry, so a crashed loader cannot block the key forever, and random jitter on TTLs keeps hot keys from expiring together.

Take key k, read 1,000 times a second, whose reload takes 50 ms. Without coalescing, every request in those 50 ms misses: about 50 identical queries. With a lock, one request loads k, and the other 49 wait about 50 ms, then read the refilled value.

• Caching

  Cache-aside is the read path everything here builds on.

  Open lesson

Case Scenario

To see how one refill can protect a database from a burst of misses, consider the cached home page of a busy news site: the entry expires during a traffic spike, and ten thousand requests miss at once, each about to query the database. The first miss marks the key in flight and reloads it; the other 9,999 wait for that one result.

A popular “home” entry can expire while ten thousand requests are arriving. They all observe the same absence before any one response restores the value, turning one routine expiry into a sudden database overload. The service must keep those callers moving while the database performs one refill for “home,” not ten thousand identical reads.