EMZETT.Load Balancing
In short: Distributing incoming requests across several servers, so that no single server is overloaded and the overall load is carried evenly.
In more detail: A load balancer sits logically in front of a group of servers and decides according to a strategy (e.g. round robin, lowest current load, or anycast routing) which server handles a specific request. Increases both performance (parallelisation) and resilience (if one server fails, the others take over).
In Depth
Load balancers work at different OSI layers, each with its own advantages and disadvantages: a layer 4 load balancer decides purely on the basis of IP address and port (fast, but “blind” to the content of the connection); a layer 7 load balancer even looks at the HTTP content and can, for example, send requests for /api/* specifically to different servers than requests for /static/* — more flexible, but more compute-intensive.
Typical distribution strategies:
Round robin - in turn, one server after another
Least connections - to the server with the fewest currently open connections
IP hash - the same client always lands on the same server (session persistence)
The choice of strategy has practical consequences: round robin is simple, but ignores the fact that some requests are much more expensive than others — a server can be overloaded despite the same number of requests. IP hash solves a different problem: if an application keeps session data locally on a server (instead of in a shared database or a cache such as Redis), a client has to land on the same server with every request (“sticky sessions”), otherwise its login status is lost.
Besides the performance gain through parallelisation, resilience is the second big advantage: a load balancer continuously checks via health checks whether a backend server still answers and automatically takes failed servers out of the rotation — ideally, users don’t notice a server failure at all, because their requests are simply routed to one of the remaining servers.