The "too small to fail" memory-allocation rule

Posted Dec 24, 2014 15:33 UTC (Wed) by dumain (subscriber, #82016)
In reply to: The "too small to fail" memory-allocation rule by ibukanov
Parent article: The "too small to fail" memory-allocation rule

The copy-on-write fork semantics don't have to lead to the OOM killer. Selecting strategy 2 in /proc/sys/vm/overcommit_memory should allow a linux box to overcommit real memory without overcommiting virtual memory. The penalty for actually using the memory then becomes swapping rather than random process death.

to post comments

The "too small to fail" memory-allocation rule

Posted Dec 25, 2014 18:50 UTC (Thu) by quotemstr (subscriber, #45331) [Link] (5 responses)

overcommit=2 is not usable in practice on general-purpose systems: MAP_NORESERVE is ignored in that mode. Many programs (like the JVM) reserve large chunks of address space on startup and internally allocate out of that. Because the kernel ignores [1] MAP_NORESERVE, these address space reservations become actual *commit charge* claims on the system and require resource allocations far in excess of what's actually needed even ignoring COW fork considerations.

MAP_NORESERVE being ignored when overcommit=2 isn't a "gotcha". It's fundamental brokenness.

[1] https://www.kernel.org/doc/Documentation/vm/overcommit-ac...

The "too small to fail" memory-allocation rule

Posted Dec 25, 2014 19:04 UTC (Thu) by fw (subscriber, #26023) [Link] (4 responses)

OpenJDK has been fixed and uses PROT_NONE mappings to reserve uncommitted address space (which is then committed with mprotect calls, as needed).

MAP_NORESERVE is not named appropriately, but it does what it does. The problem is that programmers do not know about overcommit mode 2, do not test with it, and hence never realize the need for the PROT_NONE approach.

The "too small to fail" memory-allocation rule

Posted Dec 25, 2014 19:58 UTC (Thu) by quotemstr (subscriber, #45331) [Link] (3 responses)

Agreed on the lack of testing --- but what bugs me is that MAP_NORESERVE is documented as doing the right thing, and doesn't. That means that programs written by the few well-intentioned developers are aware of overcommit issues frequently don't end up working correctly.

Sadly, the commit charge implications of MAP_NORESERVE are documented but silently broken, but the commit charge implications of PROT_NONE are undocumented and in theory mutable in future releases. All this gives mmap(2) a Rusty Russel score of around -4.

The "too small to fail" memory-allocation rule

Posted Dec 25, 2014 20:14 UTC (Thu) by fw (subscriber, #26023) [Link] (2 responses)

Which documentation? The manual page talks about error reporting through SIGSEGV, which should be sufficiently discouraging to anyone who actually reads it.

The "too small to fail" memory-allocation rule

Posted Dec 25, 2014 20:23 UTC (Thu) by quotemstr (subscriber, #45331) [Link] (1 responses)

The man page says this about MAP_NORESERVE:

Do not reserve swap space for this mapping. When swap space is reserved, one has the guarantee that it is possible to modify the mapping. When swap space is not reserved one might get SIGSEGV upon a write if no physical memory is available. See also the discussion of the file /proc/sys/vm/overcommit_memory in proc(5). In kernels before 2.6, this flag had effect only for private writable mappings.

That looks a lot like a a flag for mapping that doesn't reserve commit charge. There's no mention of the flag not actually working; there's no reference to better techniques. MAP_NORESERVE is just an attractive nuisance.

There's just tremendous confusion over exactly Linux does with commit charge even among people well-versed in memory management fundamentals. There's no clarity in the API either. MAP_NORESERVE is dangerous because it's the only publicly-documented knob for managing commit charge and it's a broken knob nobody is interested in fixing.

(And no, SIGSEGV is not something that will discourage use. Those who know what they're doing can write code that recovers perfectly safely from SIGSEGV.)

The "too small to fail" memory-allocation rule

Posted Apr 18, 2019 6:36 UTC (Thu) by thestinger (guest, #91827) [Link]

In case anyone comes across this old comment, the official documentation is at https://www.kernel.org/doc/Documentation/vm/overcommit-ac... and is accurate. It covers the rules properly.

The linux-man-pages project is not the official kernel documentation and has often been inaccurate about important things for many years. MAP_NORESERVE doesn't work as it describes at all.

It has no impact in the full overcommit mode or the memory accounting mode, which makes sense. It only has an impact in the heuristic overcommit mode, where it disables the heuristic for causing immediate failure.

Note how a read-only (or PROT_NONE) mapping with not data has no accounting cost:

For an anonymous or /dev/zero map
SHARED - size of mapping
PRIVATE READ-only - 0 cost (but of little use)
PRIVATE WRITABLE - size of mapping per instance