CPU pipeline

In short: A CPU pipeline divides the execution of an instruction into several stages that, like an assembly line, work simultaneously on different instructions. This way, ideally one instruction finishes per cycle even though each needs several cycles.

In more detail: Instead of finishing one instruction completely before the next begins, the CPU overlaps the steps. While instruction 1 is executing, instruction 2 is already being decoded and instruction 3 fetched. This raises throughput and allows higher clocks because each stage has less work.

In Depth

Classic 5 stages (RISC):

Stage	Task
IF (instruction fetch)	Fetch instruction from memory/cache
ID (instruction decode)	Decode instruction, read registers
EX (execute)	Compute in the ALU
MEM (memory)	Read/write memory
WB (write back)	Write result to register

Cycle:   1   2   3   4   5   6   7
Instr 1: IF  ID  EX  MEM WB
Instr 2:     IF  ID  EX  MEM WB
Instr 3:         IF  ID  EX  MEM WB

After ramp-up, from cycle 5 one instruction finishes every cycle.

Problems (hazards):

• Data dependency: An instruction needs the result of the previous one. Solution: forwarding or a short pause (stall).
• Structural hazard: Two stages need the same unit at once.
• Control hazard (branches): At a branch you do not know which instructions come next. Branch prediction guesses. On a wrong guess the wrongly loaded instructions are discarded (“pipeline flush”, loses 10 – 20 cycles).

Developments:

• Superscalar: Several pipelines in parallel, several instructions per cycle.
• Out-of-order execution: Instructions are reordered as soon as data is ready.
• Deep pipelines: The Intel Pentium 4 had 31 stages, which brought high clocks but costly mispredictions. Modern cores have about 14 – 20.

Why important? Pipelines explain why code with many unpredictable branches is slower and why IPC (instructions per cycle) matters more than just GHz.

See also: Out-of-order execution, CPU structure: execution units and cache, x86-64 and ARM64