SIMD and vectorisation

In short: SIMD (“single instruction, multiple data”) lets a single CPU instruction process several values at once.

In more detail: A wide register (128, 256 or 512 bits) holds, for example, eight 32-bit numbers. One addition computes all eight at once. This speeds up image, audio and matrix computation greatly.

In Depth

• Instruction sets: SSE and AVX on x86, NEON and SVE on ARM, RVV on RISC-V.
• Auto-vectorisation: Compilers turn simple loops into SIMD code by themselves when iterations are independent.
• By hand: Via “intrinsics” (special functions) or libraries. More laborious, but controllable.
• Brakes: Unaligned memory access, branches inside the loop, data dependencies, loops that are too short.
• Related: GPUs take this principle much further (SIMT). See AI chips compared.
• Power: Very wide units can lower the clock (AVX-512 throttling on some CPUs).

See also: CPU structure