Distributed Counters with Atomic Increments

Foundations

So far, each limit lived in one place. Now, what if the same caller's requests are spread across many server processes, so a count kept inside each process sees only its own share?

Keep the authoritative count in a shared store, under a key such as (caller, route, window), and update it with one atomic increment that returns the new value: allow if it is within the limit. Separate read and write steps would let two servers both pass the last allowed request. Give each key an expiry at the end of its window, and let each process keep a loose local prefilter that sheds obvious floods without granting anything.

Take a limit of 3 per minute and one caller's requests arriving on three different servers. Counted per server, each sees at most 2, so all 6 requests pass. Counted with one shared atomic increment, the fourth request reads 4 and is denied, whichever server handled it.

• Caching

  The read path sits behind cache-aside.

  Open lesson

Case Scenario

To see how one shared counter enforces a limit across many servers, consider an edge gateway like Kong or Envoy in front of many services: routes allow different amounts of traffic, and clients are untrusted. Every gateway process atomically increments the same (user, route, minute) key, so read allows 3 per minute and export 1, however many processes run.

One edge gateway protects routes with different costs and runs as several processes. Per-process counts multiply every route’s quota, while one hot client can consume gateway CPU and shared-store bandwidth before the real check runs. The gateway must enforce read = 3 and export = 1 across all processes, while shedding an obvious local flood early.