News feed

What you are designing

A logged-in user opens the app and sees a page of posts from accounts they follow, newest or ranked, and can page down. Posting is much rarer than reading. Some accounts have millions of followers. Assume hundreds of millions of users and that the feed should render in a couple of hundred milliseconds.

Requirements

• Publish a text post (media is a pointer, not the hard part).
• Follow and unfollow.
• Read a paged home feed.
• Delete a post and have it disappear on a short bound.

Ranking quality is a deep dive, not a reason to skip the candidate path. Availability beats a perfectly consistent feed: missing a post for a second is acceptable, showing a deleted post for an hour is not.

Entities and API

User, Follow(follower, followee), Post, Timeline(user_id, post_id, score).

POST /v1/posts          { text }
POST /v1/follows        { followee_id }
GET  /v1/feed?cursor=

Identity comes from the session. The feed is never cached as one anonymous object.

High-level design

flowchart LR
  W[Post API] --> Posts[(Posts)]
  W --> Fan[Fan-out workers]
  Fan --> TL[(Timeline ids)]
  R[Feed API] --> TL
  R --> PC[(Post cache)]
  R --> Rank[Rank page]

The post row is the source of truth. Workers append the post id to followers’ timelines when the author is under the celebrity threshold. The read path loads a window of ids, hydrates posts from cache, drops ones the user should not see, and returns a cursor.

Data model

CREATE TABLE posts (
  id           BIGSERIAL PRIMARY KEY,
  author_id    BIGINT NOT NULL,
  body         TEXT NOT NULL,
  created_at   TIMESTAMPTZ NOT NULL DEFAULT now(),
  deleted_at   TIMESTAMPTZ
);
CREATE INDEX posts_author_time ON posts (author_id, created_at DESC);
CREATE TABLE follows (
  follower_id BIGINT NOT NULL,
  followee_id BIGINT NOT NULL,
  PRIMARY KEY (follower_id, followee_id)
);
CREATE INDEX follows_followee ON follows (followee_id, follower_id);

The second follow index serves fan-out (“who follows this author?”). Timelines at scale live in Redis or a wide-column table keyed by user id, not as a SQL table you page with OFFSET.

Deep dives

Hybrid fan-out. Under a threshold, push the id to each follower. Over it, record the author as a celebrity and, at read time, merge “recent ids from celebrities I follow” with the precomputed timeline. Cap how many celebrities you merge. See the fan-out page. Unfollow deletes future delivery; you can lazily skip old ids that are no longer followed.

Hydration and deletes. Cache post_id -> post with a TTL and delete the key when the author edits or deletes. The timeline can still contain the id; hydration sees a tombstone and skips it. That is cheaper than scrubbing every timeline.

Ranking. Candidate generation stays the fan-out. Take a few hundred ids, score them with features you can fetch quickly (age, author affinity stored on the follow edge, a precomputed quality score), sort, and return the page. Training the model is offline. Do not call a large language model on the feed request.

Write amplification. Fan-out reads followers in pages and writes timelines in batches, on a dedicated queue, with the celebrity path excluded. A backlog means feeds are late, so you alert on it and you shed by pausing fan-out for the largest remaining accounts first.

Levels

• Mid-level. Post, follow, and a feed that is not a full scan. A cache of posts. Pagination by cursor.
• Senior. Push versus pull with a celebrity hybrid, delete invalidation, and a cursor that is stable.
• Staff. Ranking versus generation, hot-author isolation, multi-region timelines, and what you precompute so the request stays inside budget.