Little’s law and capacity planning

In short: Little’s law states: the average number of requests in the system L equals the arrival rate λ times the average time in the system W (L = λ · W). This allows simple capacity estimates.

In more detail: If 200 requests per second arrive and each takes 0.05 s on average, 10 requests are in progress at the same time. If a server handles at most 8 at once, it is not enough. The rule holds regardless of distribution and order.

In Depth

• Utilisation: As utilisation approaches 100 %, queues and latency grow sharply (non-linearly).
• Rule of thumb: Stay below about 70 % utilisation to absorb peaks.
• Latency percentiles: P99 rises much faster under load than the average.
• Queueing theory: M/M/1 models show the rise at high utilisation.
• Capacity planning: Load tests, growth forecasts, headroom for failures (N+1).
• Autoscaling: Reacts with delay, so plan headroom and warm-up time.

See also: Amdahl’s law and scaling, SLI, SLO and error budget