Fanout-on-Write (Push Model)

Foundations

In the first lesson, every page view repeated the merge. Now, what if reads vastly outnumber writes, so it is cheaper to do the work once, when an item is published?

In fanout-on-write, the push model, publishing an item enqueues a job; a worker looks up the publisher's followers and prepends the item's id to each follower's precomputed timeline, trimming it to a fixed length. A read is then one range read of the reader's own timeline. Writes cost one insert per follower, done in the background, and an insert keyed by (reader, item) makes retries harmless.

Take a publisher with 200 followers. One publish costs 1 durable write plus 200 background timeline inserts; each follower's page view is then 1 read, however many sources they follow. A follower who opens home before the worker reaches them sees the previous page, within the freshness window.

• Queues

  Slow work leaves the request path through a queue.

  Open lesson

Case Scenario

To see how fanout-on-write makes reads cheap, consider a news feed like Instagram's: a user with 200 followers posts, those followers' home timelines should include it within a few seconds, and the post call should return quickly. A worker prepends the post to 200 timelines after the post commits, so home is one read.

A normal author has about 200 followers, and each follower expects the post in home within a few seconds. Updating all 200 homes before replying makes author latency depend on follower count, while doing no preparation makes every reader repeat the merge. The post call must return after one durable write, without waiting for 200 timeline updates.