Caching

A cache stores a derived copy closer to the reader. It is not a database. The moment you forget that, you start debugging “the user updated their name and half the fleet still shows the old one.”

Where

• Client and CDN. Great for bytes and for responses that are the same for many users. Useless for a personalized feed unless the cache key includes the user, at which point the hit rate collapses.
• In-process. Nanoseconds, and invisible to the other servers. Use it for tiny, rarely changing config. Remember that every process has its own copy.
• External (Redis, Memcached). The shared cache you draw on a whiteboard. Network hop measured in fractions of a millisecond inside a region.

Policies

Cache-aside: the app reads the cache, and on a miss reads the database and fills the cache. The database remains the source of truth. This is the default you should propose.

Read-through: the cache itself loads on a miss. Same idea, different owner.

Write-through: writes go to the cache and the database before you ack. Reads are warm. Writes are slower, and you still need a plan for cache failure.

Write-behind: you ack when the cache has it, and flush to the database later. Fast, and you can lose data. Say that out loud if you propose it. It is acceptable for view counts, not for payments.

Eviction

LRU is the usual policy. LFU is better when a large scan would otherwise flush the hot set. TTL is how you bound staleness even if invalidation fails. Size the cache from the working set, not from the whole dataset. A cache of everything is just a second database with fewer features.

The problems worth saying first

Stampede. A hot key expires. Every request misses. Fix: a lock or single-flight so one request rebuilds, and serve the previous value while that happens (stale-while-revalidate). Jitter TTLs so a million keys do not expire on the same second.

Invalidation. After a successful commit, delete the key. A racing read may refill the old value; a short TTL limits the damage. Do not try to update the cached object field-by-field unless the object is simple and the update is the only writer.

Hot keys. Replication of the key (read replicas) helps reads. It does not help if the key is also written constantly. Then you shard the logical key (key#0 … key#N) and merge on read, or you stop caching something that changes every request.

Poison. Do not store a 500. Do store a short-lived “absent” for ids that 404, if absent ids are common and creation is rare.

flowchart LR
  C[Client] --> API
  API --> Cache[(Redis)]
  API --> DB[(Postgres)]
  Cache -.->|miss| DB