Caching strategies

In short: A cache stores results so they need not be computed or loaded again every time. That makes applications faster and cheaper. The hard part is getting rid of outdated data in time.

Where caching happens

1. Browser: store files and responses locally (Cache-Control header).
2. CDN: copies near the user worldwide.
3. Server/application: frequent results in memory (e.g. Redis, memoisation: Memoization).
4. Database: buffers for pages and queries.
5. CPU: buffers in the processor (Caches).

Patterns

• Cache-aside: the application looks in the cache first, on a miss in the database and stores the result. Most common.
• Write-through: writing goes to cache and database at once. Consistent, a bit slower.
• Write-back: first only into the cache, later collected into the database. Fast, but a crash risks loss.

The hard problem: invalidation

“There are only two hard things in computer science: cache invalidation and naming things.” Ways:

• TTL (time to live): the entry expires after X seconds. Simple, but temporarily outdated.
• Delete specifically when data changes (cache tags, see cache tags and revalidateTag (Next.js)).
• Versioned names for files (app.4f3a.js) that never go stale.

Traps

1. Cache stampede: many requests fetch the same expired value at once. Remedy: locks or renewing silently a little earlier.
2. Caches that are too large displace important things (memory management and the OOM killer).
3. Personal data does not belong in shared caches.

See also: Scaling