VXLAN and overlay networks

In short: An overlay network lays a virtual network over an existing one. VXLAN wraps Ethernet frames in UDP packets, connecting machines in separate networks as if they stood on the same switch.

In more detail: In data centres and clouds, virtual machines and containers need their own isolated networks, and they move between hosts. Classic VLANs (4094 IDs) are not enough for that.

In Depth

• Encapsulation: The original frame is put into a UDP packet (port 4789) with a VXLAN header (24-bit network ID, over 16 million) and sent across the normal IP network (the “underlay”).
• VTEP: The endpoints that wrap and unwrap sit in hypervisors or switches.
• Advantages: Stretch networks across router boundaries, move VMs without IP changes, separate tenants cleanly.
• Cost: The extra header (about 50 bytes) reduces the usable packet size. The MTU must be adjusted.
• Related: GRE, Geneve and WireGuard tunnels. Kubernetes network plugins (Flannel, Calico) often use VXLAN.
• Alternative: Pure routing without encapsulation (BGP), see BGP.

See also: tunnels