Fanout-on-Read (Pull Model)

Foundations

A feed shows a reader the newest items from the sources they follow. In fanout-on-read, also called the pull model, nothing is prepared when a source publishes; each read fetches the recent items of every followed source and merges them, newest first, into one page. Publishing is a single write, and the page is always fresh.

The merge is the k-way merge from the Heap lessons: keep the front item of each source's sorted list in a heap, repeatedly take the newest, and stop when the page is full, never sorting everything. The cost is reads: every page view touches every followed source, so a reader following 500 sources pays 500 fetches per view.

Take three sources with items [x9, x4], [y8, y7], and [z6, z1], newest first. The merge compares the fronts x9, y8, and z6 and takes x9; then y8, then y7, then z6, filling a page of four without touching x4 or z1. That is three fetches here, and 500 for a heavy reader.

• Requirements

  Each worked system starts from a scoped list of verbs and numbers.

  Open lesson

• Queues

  Slow work leaves the request path through a queue.

  Open lesson

Case Scenario

To see how fanout-on-read builds a feed, consider a text home timeline like X's: users post short text, and home shows the latest posts from accounts they follow, newest first, paged. Pull merges each followed account's recent posts on every home view: cheap to post and always fresh, but one read per followee every time.

Home must show the newest text posts from followed accounts, in stable pages. A reader may follow 500 accounts, and opening home is far more common than publishing, so work repeated on every open can dominate the system even when each merge is computationally small. The finished feed must make home a range read on one user’s timeline, not rebuild it from hundreds of account histories.