Caching

A cache keeps copies of frequently read data somewhere much faster than the original — memory instead of the database — so repeat reads are cheap.

App · cache · database

App

App · cache · database

Cache · 3 slots · ~1 ms

  1. empty

Database · ~50 ms

  • user:1Asha
  • user:2Ravi
  • user:3Neha
  • user:4Kiran

The app checks the cache first (~1 ms). On a miss it reads the database (~50 ms) and keeps a copy. The cache holds 3 entries; when full, the least recently used one goes.

Step 1 / 20
Hit rate
—
Average read time
—

Step by step

The example above, written out — the same steps the animation plays.

  1. 1App · cache · database. The app checks the cache first (~1 ms). On a miss it reads the database (~50 ms) and keeps a copy. The cache holds 3 entries; when full, the least recently used one goes.
  2. 2GET user:1. Is it in the cache?
  3. 3Miss — 50 ms. Read "Asha" from the database and keep a copy.
  4. 4GET user:2. Is it in the cache?
  5. 5Miss — 50 ms. Read "Ravi" from the database and keep a copy.
  6. 6GET user:1. Is it in the cache?
  7. 7Hit — 1 ms. "Asha" straight from memory; user:1 becomes the most recently used.
  8. 8GET user:1. Is it in the cache?
  9. 9Hit — 1 ms. "Asha" straight from memory; user:1 becomes the most recently used.
  10. 10GET user:3. Is it in the cache?
  11. 11Miss — 50 ms. Read "Neha" from the database and keep a copy.
  12. 12GET user:2. Is it in the cache?
  13. 13Hit — 1 ms. "Ravi" straight from memory; user:2 becomes the most recently used.
  14. 14GET user:4. Is it in the cache?

…and 6 more steps — press Play above to watch them all.

What's happening?

  1. Cache-aside: check the cache; on a hit return it, on a miss read the database and store a copy.
  2. A full cache evicts something to make room — LRU drops the entry used least recently.
  3. Writes must update or invalidate the cached copy, or readers get stale data.

Complexity

Time
O(1) get and put for an LRU cache (hash map + linked list)
Space
Bounded by the cache size

Where you'll meet it

Redis or Memcached in front of databases, browser and CDN caches, and in-process caches for configuration and sessions.

Common mistake

Caching without an invalidation plan. "There are only two hard things in computer science: cache invalidation and naming things."

FAQ

What is a good hit rate?

It depends on the data, but hot read paths often reach 90%+. A low hit rate means the cache is too small or the data isn't reused.

Invalidate or update on write?

Invalidating (delete the key) is simpler and safer; the next read refills it. Updating can race with concurrent writes.

Why also set a TTL?

As a safety net: even if an invalidation is missed, stale data expires on its own.