mmap: mapping files into memory

In short: With mmap the operating system maps a file into a program’s address space so it can be read and written like an array in memory.

In more detail: The program no longer makes read calls. On the first access to a page, a page fault triggers loading from disk.

In Depth

• Call: mmap(NULL, length, PROT_READ, MAP_SHARED, fd, 0) returns a pointer. In Python: mmap.mmap(...).
• Advantages: No copy between kernel and program memory, convenient random access, several processes share the same pages (shared memory).
• Anonymous mmap: Without a file, this is how memory allocators (malloc for big blocks) request memory.
• Pitfalls: Accessing a truncated file raises signals (SIGBUS). With many small files or sequential reads, normal reading is often just as fast.
• Use: Databases (LMDB, formerly MongoDB), executables and libraries are themselves loaded via mmap, large model files in AI.
• Page cache: The data lives in the kernel’s page cache and is evicted under memory pressure.

See also: Copy-on-write