Caching is one of the most powerful tools in system design. A well-placed cache can cut latency by 100x and reduce database load by 99%. But caches introduce complexity: stale data, invalidation challenges, and coordination headaches. Learn when to use them, where to put them, and how to avoid the pitfalls.
A cache is fast, temporary storage. When data is requested, the system first checks the cache. If the data is there (cache hit), return it instantly. If not (cache miss), fetch from the slow source (database, API, disk) and store it in the cache for next time.
1. User requests data 2. Check cache → NOT FOUND (miss) 3. Query database → 500ms 4. Store result in cache 5. Return to user Total time: ~500ms Next request (if cache hit): ~5ms
The percentage of requests that find data in the cache. Even modest hit rates pay huge dividends:
Is this data accessed frequently? Caching only makes sense for hot data—data accessed many times. If you cache data accessed once every hour, you're wasting memory.
Different caching patterns suit different scenarios. The two most common are cache-aside (lazy loading) and write-through.
The application checks the cache. On miss, loads from the database and populates the cache.
GET user:123 1. Check Redis → MISS 2. Query PostgreSQL 3. Cache result in Redis 4. Return to client GET user:123 (again) 1. Check Redis → HIT 2. Return immediately
On write, update both the cache and the database. Reads always hit the cache.
UPDATE user:123 1. Write to Redis 2. Write to PostgreSQL 3. Return success (both done) GET user:123 1. Check Redis → HIT (always) 2. Return immediately
Write-Behind (Write-Back)
Write to cache immediately, asynchronously write to database. Fast writes, but risk of data loss if cache fails before flushing.
Refresh-Ahead
Before data expires, preemptively refresh it in the background. Avoids cold starts but requires knowing access patterns.
Caches have limited memory. When full, old data must be evicted to make room for new data. Which data should you remove? The answer depends on your eviction policy.
LRU (Least Recently Used)
Evict the item that hasn't been accessed recently. Most common choice. Works well when access patterns are temporal—recently accessed data is likely accessed again soon.
LFU (Least Frequently Used)
Evict the item accessed least often. Better for workloads where some data is accessed far more than others (e.g., popular posts vs obscure pages).
FIFO (First In, First Out)
Evict the oldest item. Simple but naive—doesn't consider access patterns.
Random
Evict randomly. Sometimes performs surprisingly well and has low overhead.
Cache size: 3 items Access pattern: A, B, C, A, D State after each: [A] → Cache: [A] [A, B] → Cache: [A, B] [A, B, C] → Cache: [A, B, C] [B, C, A] → Cache: [B, C, A] (A moved to end) [C, A, D] → Cache: [C, A, D] (B evicted - least recent) LRU timestamp: A > D > C
Phil Karlton said: "There are only two hard things in Computer Science: cache invalidation and naming things." He was only half joking.
When data in the database changes, the cache becomes stale. Serving stale data can cause inconsistencies, bugs, or data corruption.
Nightmare Scenario:
User updates their address. The database is updated. But their old cached address is served to the shipping service. Package arrives at the wrong place.
TTL (Time-To-Live)
Expire data automatically after N seconds. Simple but can serve stale data for up to N seconds. Good for data that's okay to be slightly outdated (e.g., tweet counts, article views).
Explicit Invalidation
When data changes, immediately delete it from the cache. Guarantees freshness but requires coordination (delete from cache AND database in sync).
Event-Driven Invalidation
Use message queues or event streams. Database emits "address_updated" event → cache service listens and deletes relevant entries. Decoupled and scalable.
Versioning / Tags
Instead of deleting, create a new version. Use cache tags to group related entries. Invalidate by tag rather than individual keys.
Fast staleness: TTL is simple but can serve old data.
Immediate freshness: Explicit invalidation is complex and prone to consistency bugs (what if the delete fails?).
In practice: Most systems use a combination. TTL as a safety net, explicit invalidation for critical data.
Caches can exist at multiple layers. Each layer solves different problems.
Browser caches (localStorage, sessionStorage) or mobile app caches. Eliminates network round-trips entirely.
localStorage.setItem("user", JSON.stringify(user))
// Next time
const cached = JSON.parse(localStorage.getItem("user"))Best for: Static assets, user preferences, session data
In-process cache (HashMap, LRU cache) within your application. Fast (same process), but not shared across instances.
// In-process HashMap
const cache = new Map()
function getUser(id) {
if (cache.has(id)) return cache.get(id)
const user = db.query(id)
cache.set(id, user)
return user
}Best for: Hot data, expensive computations, within single instance
Separate cache layer shared by all servers. Solves the single-instance problem but adds network latency.
GET user:123 App1: Redis miss → query DB → set Redis App2: Redis hit → return instantly App3: Redis hit → return instantly
Best for: Shared state, multiple instances, high scalability
The database caches query results and table pages in memory. Automatic, no code needed. But you don't control it.
Best for: Automatic optimization, zero overhead
User browser ↓ (static assets cached) CDN ↓ (edge cache hit?) Client application ↓ (in-process cache hit?) Redis (distributed cache) ↓ (cache miss → hit database) Database (table page cache)
Cache Simulation
Change cache size, request pattern, and eviction policy to watch hit rate and cache content evolve.
Cache capacity
4 keys
Requests served
0
Hit rate
0.0%
Current request
Cache contents
Yesterday’s most recent requests
Browser Cache (LocalStorage)
Static assets, user prefs, session data. Don't use for sensitive data.
In-Memory Map (HashMap)
Hot data within a single instance. Simple, fast, but not shared.
Redis / Memcached
Distributed cache shared by all servers. Best for scaling, but adds complexity.
CDN Cache
Edge caching for static content. Geographic distribution, fast globally.