Preventing atomic-context violations in Rust code with klint

Posted Nov 20, 2023 21:03 UTC (Mon) by NYKevin (subscriber, #129325)
In reply to: Preventing atomic-context violations in Rust code with klint by tialaramex
Parent article: Preventing atomic-context violations in Rust code with klint

Well, there's two sides to it:

* Rust's soundness guarantees do not cover leaks at all. As far as Rust is concerned, leaking memory (and never running its destructor, if any) is 100% sound in all cases. This is why mem::forget() and Box::leak() are safe functions. The soundness rules only require that *if* a destructor is run, then the object must not be used once the destructor returns. This can have some unintuitive consequences. For example, leaking a MutexGuard is sound, which means it is sound to lock a Mutex and never unlock it (and deadlock/starve all of the other threads that want to use that lock). The same applies to most of the other things in std::sync.
* If you're writing a kernel, you really really really do not want to leak memory (unless you are e.g. constructing some static data structure at boot time). Leaking memory in userspace is bad, but the kernel will clean up after you when your process terminates. Memory leaked in kernelspace will remain leaked until the system reboots. OTOH, as described in the previous bullet, there are a lot of other things you can do that Rust considers sound, but are highly undesirable in practice.

Soundness was never meant to be the same thing as correctness. It was meant to prevent UB. Leaking memory does not risk UB, so it is sound. Deadlock is not UB either, so it is sound. Etc.

to post comments

Preventing atomic-context violations in Rust code with klint

Posted Nov 20, 2023 23:22 UTC (Mon) by dvrabel (subscriber, #9500) [Link] (5 responses)

Leaking memory is only non-UB behaviour if your program has access to unlimited memory, otherwise the program will panic at some undefined point in time when it runs out of memory, although I suppose rust claims this behaviour as well-defined since the outcome when it happens is deterministic.

Preventing atomic-context violations in Rust code with klint

Posted Nov 21, 2023 0:54 UTC (Tue) by tialaramex (subscriber, #21167) [Link]

But this is exactly why I think "leaks" are a red herring. You've gone from "leak" to "unbounded memory allocation" and now there's a big problem, but the two are only related colloquially.

The sort of leaks that leakdice is written to investigate count as both and are surely a bug which must be fixed. But when you try to write down formally what isn't OK, you discover that the leak wasn't the problem, it was the unbounded thirst for memory.

"We can optimise this movie player by loading all the quarter million frames into RAM and then flipping through them" blows up just as badly on a 1990s PC with 4MB of RAM regardless of whether you wrote code to free all those images at the end, we're never getting that far.

And yes, it's not UB to drop dead once you're out of other options. For an OS kernel rebooting might be a reasonable option, and so might power off, neither of which is Undefined Behaviour.

Preventing atomic-context violations in Rust code with klint

Posted Nov 21, 2023 1:08 UTC (Tue) by excors (subscriber, #95769) [Link] (1 responses)

I don't think panicking is considered non-UB just because of determinism. Rust's goal is that safe code cannot trigger the set of behaviours it considers UB (assuming there are no soundness bugs in `unsafe` code) - that's fundamental to the relationship between safety, soundness, and UB - but safe code can trigger panics, so panics cannot be considered UB.

And I think safe code is allowed to panic for ergonomic reasons: in most applications and libraries, it would be very annoying if every object allocation and every array access and every unwrapping of an Option etc had to explicitly handle errors or propagate the errors through all the APIs in its call stack. Those operations are frequently done where it's obvious they can't fail - e.g. allocation will always succeed in default Linux userspace (it just may attract the ire of the OOM killer some time later), and you might have already done some manual bounds checks before accessing the array, etc - so it's nice to be able to skip the useless error handling code there. In the rare cases where the thing you thought was 'obvious' was actually wrong, it will panic and cleanly terminate the process, and the system can recover by e.g. restarting the application, which isn't perfect but it seems a worthwhile tradeoff.

That's a worse tradeoff in kernel code, where it's much harder to recover from termination and you'd be willing to put a lot more effort into writing panic-free code, and maybe it could make sense to declare panic as UB so the compiler guarantees safe code won't panic; but much of Rust's standard library was designed to panic, so changing the definition of UB would be too much divergence from regular Rust, and it's probably easier to just treat panics like any other non-statically-checked bug that you want to avoid.

Preventing atomic-context violations in Rust code with klint

Posted Nov 21, 2023 9:58 UTC (Tue) by tialaramex (subscriber, #21167) [Link]

The Rust standard library isn't available in Rust-for-Linux, only core (some of which is necessary to have Rust even work as a language e.g. you can't have for loops without core::iter::IntoIterator) is provided, plus their own take on alloc, but not std.

Even for writing very low level code, "This won't happen, don't bother me about it" is ergonomically necessary. It's fine if the reaction when you're wrong can be very dramatic, right down in the machine code if we have a fault but the CPU has for example not been told how to deal with any faults (or that the handlers for all faults are gone) it will just reboot.

Preventing atomic-context violations in Rust code with klint

Posted Nov 21, 2023 15:58 UTC (Tue) by MarcB (subscriber, #101804) [Link]

Something potentially happening at an undefined (better: unpredictable) point in time does not make it undefined behaviour.

Undefined behaviour is generally limited to things that are undefined within a given language specification. Here the specification will most likely say that memory allocations can fail and what happens if they do.

UB does not include things that are influenced by external factors. In this example, it might as well be another program that consumed all memory and causes you program's allocation to fail. So if your program panics or not does not even depend on its own quality.

Preventing atomic-context violations in Rust code with klint

Posted Nov 22, 2023 11:08 UTC (Wed) by farnz (subscriber, #17727) [Link]

The behaviour is well-defined, not UB, because the behaviour of Rust code in an OOM situation is well-defined; either a panic if you use the infallible allocation APIs (which assume that you can't have OOM), or a failure if you use a fallible allocation API (such as Vec::try_reserve.

This means that I can read Rust code, and I know exactly what will happen on OOM, and where OOM can happen (assuming I'm running on an underlying allocator that reports OOM, and not an overcommitting allocator that kills the process on OOM). In contrast, if OOM is UB, then once I hit an OOM situation, the program can behave in any fashion - including "time travel" back to before OOM happened followed by weird behaviour. In practice, UB is a huge problem when the compiler is doing any form of optimization, because the compiler can assume that UB does not happen, and reason from there.