Worked problems · YouTube

Worked problems

YouTube

Upload a video, transcode it into streamable segments, and play it near the viewer without pushing bytes through your API.

3 min read system-design

What you are designing

A creator uploads a video. Viewers around the world press play and get a smooth stream. They can browse a home page. Assume videos are large, watches vastly outnumber uploads, and a popular video is many millions of viewers. Comments and a full social graph are out of scope unless you have time; say so.

Requirements

  • Resumable upload and a processing state the creator can see.
  • Playback that adapts to bandwidth and starts within a couple of seconds.
  • A video page with title, owner, and a view count that is approximately right.
  • Ability to make a video private or remove it.

Playing the video is the availability-critical path. A slightly stale view count is fine.

Entities and API

Video(id, owner, state, manifest_url), Rendition, ViewCounter as an aggregate, not a row you lock.

POST /v1/videos                      -> id + multipart upload
POST /v1/videos/{id}/complete
GET  /v1/videos/{id}                 -> metadata + manifest URL if ready

The player then talks to the CDN, not to you, for segments.

High-level design

flowchart LR
  Creator --> API
  Creator --> Blob[(Original in object storage)]
  API --> Jobs[Transcode queue]
  Jobs --> Seg[(Segments + manifest)]
  Player --> CDN
  CDN --> Seg
  API --> Meta[(Video metadata)]

Complete enqueues a job (see long-running tasks). Workers produce several bitrates of HLS or DASH, write segments as immutable objects, and publish a manifest. The video row flips to ready with the manifest URL. Metadata reads are cached. Bytes are not.

Data model

CREATE TABLE videos (
  id           BIGSERIAL PRIMARY KEY,
  owner_id     BIGINT NOT NULL,
  title        TEXT NOT NULL,
  state        TEXT NOT NULL, -- uploading | processing | ready | removed
  manifest_key TEXT,
  created_at   TIMESTAMPTZ NOT NULL DEFAULT now()
);

View facts are events (video_id, ts) sampled or aggregated in a stream job to a compact counter store. The page reads the counter through a cache.

Deep dives

Playback. The manifest lists segments of a few seconds at each bitrate. The client picks a bitrate, fetches segments from the nearest CDN edge, and switches down if downloads stall. Range requests are the fallback if you did not segment; segmentation is the cleaner answer. An origin shield protects storage when a new video expires on every edge at once.

Processing cost. Transcoding is CPU-heavy and parallel by chunk, then stitched. It does not belong on the web tier. Show the creator progress from the job row. A failure leaves the video in processing with an error, not a half-published manifest.

Takedown and privacy. Manifest URLs are unguessable and short-lived or short-cached. Setting state to removed stops the API from handing out the manifest and purges the manifest cache. Segments can be immutable and cached long; without the manifest they are not useful. Signed URLs if the video is private, checked before the CDN will serve.

Popularity. The home page reads precomputed recommendation ids. A trending counter is a stream window, not a scan. The deep dive if they push is hot-video CDN hit rate, not a cleverer database.

Levels

  • Mid-level. Direct-to-storage upload, async transcode, CDN playback, metadata in a database.
  • Senior. Adaptive segments and manifest, resumable parts, view counts off the play path, takedown via the manifest.
  • Staff. Cost of renditions, origin shield, regional caches, and a recommendation path that can fail independently of playback.

Further reading

Check yourself

Answer the questions. Multiple choice is scored on this device only.

1. What does the player fetch from the CDN?

2. A takedown must stop playback quickly. You should:

3. Where do recommendations run?