Worked problems
URL shortener
Create a short link and redirect fast. The interesting parts are id generation, cache, and abuse, not the CRUD.
What you are designing
A service where a client submits a long URL and receives a short code, and anyone hitting that code is redirected to the long URL. Assume a few thousand creates per second at the peak you care about, and tens of times that in redirects. Links live for years. Custom aliases are optional. Analytics can lag.
Requirements
- Create a short link, optionally with a custom alias and an expiry.
- Redirect with a low latency budget, on the order of tens of milliseconds at the edge.
- Disable a link and have redirects stop within a short bound (a minute, not a week).
- Do not require an account for the basic flow. If you add accounts, they are for managing links, not for the redirect.
Availability of redirects matters more than linearizability. A brand-new link must work for its creator immediately.
Entities and API
Link(code, long_url, owner_id, created_at, expires_at, disabled) and, if you count, a separate click stream.
POST /v1/links { url, alias? } -> 201 { code, short_url }
GET /v1/links/{code} -> 302 Location, or 404
DELETE /v1/links/{code} -> 204
The redirect is the hot path and should stay cacheable. Do not put the creator’s identity on it.
High-level design
flowchart LR
C[Client] --> API[Link API]
API --> DB[(Postgres)]
R[Browser] --> Edge[CDN or edge]
Edge -->|miss| API
API --> Cache[(Redis)]
Cache -->|miss| DB
API --> Log[Click log]
Create writes Postgres with a unique code, then fills the cache. Redirect reads the cache, then the database, and returns 301 or 302. 301 is aggressively cached by clients; use 302 if you need to retire links and still see clicks. Say which.
Data model
CREATE TABLE links (
code TEXT PRIMARY KEY,
long_url TEXT NOT NULL,
owner_id BIGINT,
expires_at TIMESTAMPTZ,
disabled BOOLEAN NOT NULL DEFAULT false,
created_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
The primary key is the only index the redirect needs.
Deep dives
Generating the code. A database sequence (or a range of ids handed to each app server) converted to base62 is collision-free and short. It is also guessable. If that matters, random 64-bit values encoded in base62, inserted with a unique constraint and a retry on conflict, are enough at this QPS. A cryptographic hash of the URL collides and couples one URL to one code. Pick based on the product, and say the collision behavior.
Speed. At redirect time you have one key lookup. Redis holds the hot codes. A CDN can cache the 302 for codes that are not personalized, with a short TTL so disable works. The creator’s first click should read through to the primary or trust the write response, so replica lag does not 404 a link they just made.
Abuse and edits. Allow a denylist and a disable flag. Delete the cache key on disable. Do not update the long URL in place if the short code has been shared; that is a phishing vector. Expire abandoned rows. Click analytics go to a log and a batch counter, not to an update of links.
Levels
- Mid-level. Clear API, a unique code, a database, and a cache in front of redirects. Mentions collision or sequence.
- Senior. States read/write ratio, 302 versus 301, cache invalidation on disable, and async analytics. Discusses guessability.
- Staff. Abuse, multi-region redirects, how the id generator behaves when a region is partitioned, and what you refuse to build (synchronous global click counts).
Further reading
Check yourself
Answer the questions. Multiple choice is scored on this device only.
1. Redirect QPS is 50× create QPS. Where do you spend the design?
2. A hash of the long URL as the code will:
3. How do you count clicks without slowing the redirect?