Memory management and the OOM killer

In short: Linux actively uses free memory as a cache. If memory really runs out, the OOM killer ends a program so the system does not stall.

What free -h shows

              total   used   free  shared  buff/cache  available
Mem:           15Gi   4.1Gi  1.2Gi   300Mi        10Gi      10Gi
Swap:         2.0Gi      0   2.0Gi

• free is really unused. It is small on purpose.
• buff/cache are files in memory that the system gives back immediately when needed.
• available is what is actually available for programs. This number counts.

Swap

An area on disk (partition or file) for rarely used memory. It prevents bottlenecks but is much slower. Today there is also zram, compressed swap in RAM. The setting vm.swappiness decides how willingly things are swapped out.

The OOM killer

When neither free memory nor cache nor swap suffice, the kernel picks a process by a score (oom_score) and ends it. The logs then say Out of memory: Killed process ..., viewable with journalctl -k or dmesg, see Logs and scheduling.

Services under systemd and containers can get fixed limits, see Containers on Linux. Helpful: find the memory hog with top or ps aux --sort=-%mem | head instead of just buying more RAM.

See also: Stack and heap, Memory leak