L1, L2 and L3 cache

In short: The cache is a very fast buffer in the CPU that holds data and instructions that will be needed soon. There are three levels: L1 (small, fastest), L2 (medium) and L3 (large, shared, slower).

In more detail: Main memory is very slow for a CPU: one access takes as long as hundreds of computation steps. The cache bridges this with the observation that programs need the same data repeatedly (temporal locality) and neighboring data (spatial locality). Several levels are a compromise: the faster a memory, the more expensive and smaller it is.

In Depth

Level	Typical size	Access time	Belongs to	Special
L1	32 – 64 KB (each data and instructions)	~4 cycles	each core	Fastest and smallest, split into L1d (data) and L1i (instructions)
L2	256 KB – 2 MB	~12 cycles	usually each core	Buffer between L1 and L3
L3	8 – 128 MB (and more, 3D V-Cache)	~40 cycles	all cores together	Exchanges data between cores
RAM	GB	~200+ cycles	external	for comparison

How it works:

• Data is loaded in cache lines of 64 bytes.
• Hit: The data is there, fast. Miss: Data must be fetched from the next level, the CPU waits.
• Eviction: When the cache is full, usually the longest unused line is thrown out (LRU-like).
• Coherence: If one core changes data, other cores must update or discard their copies (protocols such as MESI).

Why several levels?

• A single large cache would be slow (longer paths, more management).
• A small, fast L1 catches most accesses. What does not fit lies in L2 and L3.
• This yields a combination of high speed and enough capacity.

Practice:

• Programs run faster when they read data in order (arrays) and not in jumps (linked lists, random access).
• Games and databases benefit strongly from a large L3 (AMD X3D, 3D V-Cache).
• Cache size is, besides clock and core count, a buying criterion.
• Attacks such as Spectre and Meltdown exploited timing differences in the cache.

See also: CPU structure: execution units and cache, Memory hierarchy and cache lines, Caches