Fan-out

Fan-out is the decision of when to multiply work by the number of readers. You either pay at write time, so each reader has a private copy of “what to show,” or you pay at read time, assembling the view from shared data. Reads usually outnumber writes, which is why paying at write time is the default, until one writer has so many readers that the write never finishes.

On write

When someone posts, a worker reads their follower list (in pages, not one giant fetch) and appends the post id to each follower’s timeline, usually a Redis list or a row in a wide-column table. The home request is “give me the first page of my timeline,” which is one key. Ordering is the append order, or a score you set explicitly.

Bound it. A worker pool with a max concurrency, a per-user budget, and isolation for heavy hitters. If this queue backs up, new posts are late for everyone, so the celebrity must not share an unbounded queue with ordinary users.

On read

Store the post once. A read loads the accounts the user follows and merges their recent posts by time, with a cursor. This is simple and write-cheap. It gets expensive as the follow count grows, and it is painful if every read hits disk. Cache each account’s recent post ids.

Hybrid

Do write fan-out for accounts under a threshold (thousands of followers, pick a number and stick to it). Do read fan-out for accounts above it. A user’s feed read is “my precomputed timeline, merged with recent posts from the celebrities I follow.” Celebrity posts show up slightly later and cost a little more at read time. Ordinary posts stay cheap. You will be asked “what if I follow 500 celebrities?” Cap the merge set, or precompute for users who follow many celebrities too. Say the cap.

Deletions and edits must fan out as well, or you store only ids in the timeline and the post body is fetched through a cache that you invalidate. Prefer ids. Copying the whole post into a million timelines makes an edit a disaster.

flowchart TB
  P[New post] --> D{Follower count}
  D -->|small| W[Append id to each follower timeline]
  D -->|huge| S[Store once]
  R[Read feed] --> T[Precomputed timeline]
  R --> M[Merge celebrity posts]