Committing to Rust in the kernel

The project to enable the writing of kernel code in Rust has been underway for several years, and each kernel release includes more Rust code. Even so, some developers have expressed frustration at the time it takes to get new functionality merged, and an air of uncertainty still hangs over the project. At the 2024 Maintainers Summit, Miguel Ojeda led a discussion on the status of Rust in the kernel and whether the time had come to stop considering it an experimental project. There were not answers to all of the questions, but it seems clear that Rust in the kernel will continue steaming ahead.

Ojeda started with the topic of the flexibility needed from the kernel's subsystem maintainers. Two years ago, before the initial Rust support was pulled into the kernel, he had requested that flexibility because there would be the need to change some core APIs at times to fit Rust code in. The need for that flexibility is being felt now, he said.

There are some clear differences in the expectations around Rust in the kernel, he continued. He has read through thousands of comments and emails on recent events, and has seen a wide range of opinions on the state of the Rust-for-Linux project and where it is headed. It would be a good thing to converge on a common understanding of what the goals are. People and companies want to invest in Rust for the kernel, but they are unsure about its future.

Jason Gunthorpe said that he, like many other kernel developers, has not participated in this work so far. The project was intended to demonstrate that Rust is suitable for kernel usage; he is waiting for the decision on the outcome. Dave Airlie said that the experiment is not complete, but Greg Kroah-Hartman said that it is clear at this point that Rust in the kernel is viable. Part of the reason for the apparent slow pace of the work, he said, is that the Rust developers are concentrating on device drivers; since drivers must interface with many other kernel subsystems, there is a lot of support code that must be merged. That takes time.

Gunthorpe said that he would like to see a clear message that Rust is a success before jumping into it; he also is unable to work with Rust until a suitable compiler is available in RHEL. Airlie said that perhaps, for Gunthorpe, the time had not yet come.

Tooling and help

Arnd Bergmann said that there was no doubt that drivers written in Rust would be better than those in C, but he wondered how long it would take to merge support in all the necessary subsystems, and when the tooling would be widely available. When, he asked, will he be able to build kernel code with a Rust compiler shipped by his distribution? Ojeda answered that multiple compiler versions are now supported by the kernel code, and that suitable compilers are available from many community-oriented distributions. Airlie said that it is too soon to ask the Rust community for a completely stable compiler to build kernel code with; there just is not yet enough Rust code in the kernel to make that happen.

Linus Torvalds admonished the group that he did not want to talk about every subsystem supporting Rust at this time; getting support into some of them is sufficient for now. When Airlie asked what would happen when some subsystem blocks progress, Torvalds answered "that's my job". Christian Brauner said that the binder driver is motivating much of the subsystem work now, including the somewhat contentious filesystem abstractions. That code is being reviewed now. Airlie added that the first real driver to be merged will be a sort of inflection point, after which the pace will pick up; the next challenge after that will be the creation of Rust infrastructure that is callable from C.

Will Deacon asked Ojeda about the support that the Rust community was offering to kernel developers; Ojeda answered that he has been building a team of experts to help where needed. Some of these people are core Rust developers who know the language thoroughly; they can help to review patches even if they lack deep kernel experience.

Torvalds pointed out that there are kernel features that are currently incompatible with Rust; that is impeding Rust support overall. He mentioned modversions in particular; that problem is being worked on. The list of blocking features is getting shorter, he said, but it still includes kernel features that people need.

Managing expectations

Dan Williams pointed out that he once spent two years just getting a new mmap() flag merged. It is necessary to manage expectations on the Rust side, he said; merging all of that code will be a slow process. Ojeda acknowledged this point, but said that the companies funding the Rust work are not seeing it going upstream; that is making them reluctant to continue that funding going forward.

Brauner said that nobody has ever declared that the filesystem abstractions would not be merged; the discussion is all about the details of how that will happen.

Ted Ts'o said that the Rust developers have been trying to avoid scaring kernel maintainers, and have been saying that "all you need is to learn a little Rust". But a little Rust is not enough to understand filesystem abstractions, which have to deal with that subsystem's complex locking rules. There is a need for documentation and tutorials on how to write filesystem code in idiomatic Rust. He said that he has a lot to learn; he is willing to do that, but needs help on what to learn. (See this article for a discussion of how the Rust-for-Linux developers are working to meet this need).

Torvalds said that it is not necessary to understand Rust to let it into a subsystem; after all, he said, nobody understands the memory-management subsystem, but everybody is able to work with it. I pointed out that the Rust developers are not just creating subsystem bindings; they are trying to create inherently safe interfaces, and that often requires changes on the C side. That increases the impact on subsystems, but also makes the C code better. Airlie added that the Rust developers have to bring maintainers along with them, or the maintainers will not understand what is happening.

Deacon raised the question of refactoring on the C side. Changing C interfaces will often have implications for the Rust code and may break it; somebody will the have to fix the problems. Torvalds said that, for now, breaking the Rust code is permissible, but that will change at some point in the future. Kroah-Hartman said that the Rust developers can take responsibility for the maintenance of the abstractions they add.

Steam right ahead

Torvalds said that nothing depends on Rust in the kernel now, and nothing will for some time yet. What is important is to make forward progress, so developers should "steam right ahead" and not worry about these problems for now. It is enough to get things working, even though the details are not right. Once users are depending on Rust code, it will be necessary to worry more, he said, but kernel developers should not fail by being too careful now.

Thomas Gleixner said that the Rust developers are careful about documenting their code, and he is not frightened by the prospect of refactoring it. If he does not understand something, he will simply send an email to the developer, just as he does with C code. Torvalds added that Rust has a lot to offer, and the kernel should try to take advantage of it. Kroah-Hartman said that it could eliminate entire classes of bugs in the kernel.

Deacon asked how many developers are working on the Rust side now; Ojeda answered that there are currently six or seven people, most of whom are "real Rust experts". The strongest kernel expertise in the group had been Wedson Almeida Filho, who had recently left the project. That was a real loss, but Ojeda is working to recruit others.

Gleixner said that, 20 years ago, there had been a lot of fear and concern surrounding the realtime kernel work; he is seeing the same thing now with regard to Rust. We cannot let that fear drive things, he said. Torvalds said that Rust has been partially integrated for two years. That is nothing, he said; the project to build the kernel with Clang took a decade, and that was the same old language.

Julia Lawall asked what happens when things change on the C side; how much will leak through into the Rust code? Bergmann said that reviewing Rust abstractions for a C subsystem without knowing Rust is not difficult; he can reach a point where he understands the code, but would not feel able to change it.

Torvalds said that the community can play around with Rust for a few years. Gunthorpe, though, said that it would be good to get something into production; that would give the project some needed momentum. The binder driver might be a good choice. Ojeda said that would help to justify more support from companies. As the session closed, though, the primary outcome may well have been expressed by Torvalds, who suggested telling people that getting kernel Rust up to production levels will happen, but it will take years.

Index entries for this article
Kernel	Development tools/Rust
Conference	Kernel Maintainers Summit/2024

---

to post comments

RHEL Support

Posted Sep 24, 2024 14:20 UTC (Tue) by jgg (subscriber, #55211) [Link] (4 responses)

> Gunthorpe said that he would like to see a clear message that Rust is a success before jumping into it; he also is unable to work with Rust until a suitable compiler is available in RHEL. Airlie said that perhaps, for Gunthorpe, the time had not yet come.

To clarify my remark, I was speaking from the perspective of our community members that are working on server/enterprise/hyperscale projects where deplyoment of the project usually requires running on some older kernel via backports. Using RHEL as some shorthand for this ecosystem. This is a distinctly different set of concerns from Fedora, Android or Embedded communities.

The issue here is not that a "suitable compiler" exists in RHEL, but that all the distros have enabled Rust in their kernels, that they fully support Rust on their commercially relavent server architectures (S390 and Power! I have users!) and everything is in place to backport and consume an add-on-top "driver backport package". There are many technical gaps still to be resolved here.

This was, in part, a reaction to Miguel's earlier remark that companies were reluctant to participate. It will be very hard for a company to fund writing new code in Rust if none of their customers can run that code and they have to write a C version anyhow.

RHEL Support

Posted Sep 25, 2024 4:57 UTC (Wed) by admalledd (subscriber, #95347) [Link] (3 responses)

FWIW, in -theory- 360, POWER, Sparc, etc, should work with Rust. Just they aren't "Tier 1 officially supported" yet since they lack commercial backing/QA/validation, and getting them into the kernel would be its own minor project.

For clarity on ISA/Platform support: https://doc.rust-lang.org/nightly/rustc/platform-support....

Though the above is more about applications, not kernels, you can (mostly) just pay attention to the first bit of a platform-triple there for what ISA is supported. That page also clarifies better than I could what T1/T2/T3/etc all mean on "officially supported".

That is all for the LLVM backend, and there are of course the two projects (codegen-gccjit and GCC-rs) to use GCC in some flavor instead.

So yea, there is concern still on missing platforms vs the Linux Kernel, but the hope is the commercial interests involved with those will either (1) support LLVM to those, or (2) support one or the other GCC project.

All that said, still likely years away for any of the missing platforms to need to care about Rust: many of the subsystems and drivers likely to be written won't ever be intended for such legacy platforms anyways.

RHEL Support

Posted Sep 25, 2024 8:44 UTC (Wed) by farnz (subscriber, #17727) [Link] (2 responses)

As a really quick summary, only x86-32, x86-64 and AArch64 are in Tier 1. Tier 2 brings in AArch32, LoongArch, RISC-V, PowerPC, SPARC64 and WASM. Tier 3 then brings in AVR, C-SKY, MIPS, SPARC32, S390x (but not plain S390) and 68k.

Very roughly, the tiers are:

1. Tier 1 blocks merging of PRs if any builds or tests fail in CI.
2. Tier 2 blocks merging of PRs if any builds fail in CI; tests are allowed to fail.
3. Tier 3 simply has one or more people promising to work on fixes if it fails to build in CI.

RHEL Support

Posted Sep 25, 2024 10:28 UTC (Wed) by sam_c (subscriber, #139836) [Link]

What ends up being problematic for us is that only certain tiers get "host tools" built which means we have to build our own binaries for distributing Rust. sparc64 is an example.

RHEL Support

Posted Sep 26, 2024 12:29 UTC (Thu) by arnd (subscriber, #8866) [Link]

I also see hexagon and xtensa in the tier 3 list of the rustc manual, leaving only 7 architectures with no support at all, which are also the ones with very few users these days:

alpha, parisc and superh are old, with the latest CPU cores dating back to 2005, any work going on here is driven by hobbyists, but m68k shows that this may still happen.

arc, microblaze, and nios2 are still available for new designs but most users have moved on to risc-v here, and I don't expect anyone to put money into a rustc port. Nios2 and microblaze are likely to get phased out from the kernel in the next few years. Openrisc is somewhere between those, as a softcore that is not owned by a single company.

Building Rust-for-Linux on stable Rust

Posted Sep 24, 2024 16:31 UTC (Tue) by josh (subscriber, #17465) [Link] (22 responses)

The effort to get Rust-for-Linux to build using entirely stable Rust features (which involves stabilizing and in some cases designing several Rust features) is also progressing full steam ahead. I expect that that'll make it much easier for Linux distributions to include drivers written in Rust.

Building Rust-for-Linux on stable Rust

Posted Sep 24, 2024 18:42 UTC (Tue) by sam_c (subscriber, #139836) [Link] (21 responses)

Genuinely quite pleased to hear this as it's been causing us a lot of concern.

But I do wonder how this also isn't a sign that Rust isn't stable enough for use yet in the kernel? Nobody is adding dependencies on bleeding-edge GCC or Clang for unratified C features which may change at any point ("unstable"). Extensions do get used but that is once the compilers agree they're an interface and not subject to change, and the kernel always has fallbacks for older GCC and Clang where they're not available.

Building Rust-for-Linux on stable Rust

Posted Sep 24, 2024 18:51 UTC (Tue) by mb (subscriber, #50428) [Link] (5 responses)

>Extensions do get used but that is once the compilers agree

That's pretty much not true historically.
For most of the time Linux just used what gcc provided as an extension and didn't care about anything else or whether it was "mature enough".

>and the kernel always has fallbacks for older GCC

Distributions used to ship special versions of gcc just for the kernel.

Building Rust-for-Linux on stable Rust

Posted Sep 24, 2024 18:54 UTC (Tue) by sam_c (subscriber, #139836) [Link] (4 responses)

> For most of the time Linux just used what gcc provided as an extension and didn't care about anything else or whether it was "mature enough".

It took a very long time indeed to agree to use newer C: https://lwn.net/Articles/885941/.

> Distributions used to ship special versions of gcc just for the kernel.

That doesn't mean we want to return to that. But the minimum GCC version even now is pretty conservative anyway?

Building Rust-for-Linux on stable Rust

Posted Sep 24, 2024 23:50 UTC (Tue) by Heretic_Blacksheep (subscriber, #169992) [Link] (3 responses)

Kernel docs says minimum version is GCC v. 5.1, but it varies depending on architecture. Obviously recently ported hardware would require a newer GCC so take that with a grain of salt. If a (micro)arch was only integrated into GCC last year, that's your likely minimal version. The rest of the build tools versions vary similarly.

Building Rust-for-Linux on stable Rust

Posted Sep 27, 2024 12:26 UTC (Fri) by andy_shev (subscriber, #75870) [Link] (1 responses)

“Obviously recently ported hardware would require a newer GCC so take that with a grain of salt.” Obviously? Not at all to me! Can you elaborate on how you came to this conclusion? (Also it looks like you missed word “driver” along the lines)

Building Rust-for-Linux on stable Rust

Posted Sep 27, 2024 16:55 UTC (Fri) by knewt (subscriber, #32124) [Link]

>> “Obviously recently ported hardware would require a newer GCC so take that with a grain of salt.” Obviously? Not at all to me! Can you elaborate on how you came to this conclusion? (Also it looks like you missed word “driver” along the lines)

I imagine the thought here is that newer processor architectures will have only gotten GCC support more recently, and as such only in newer versions of GCC, which feels reasonable.

Building Rust-for-Linux on stable Rust

Posted Sep 27, 2024 16:59 UTC (Fri) by knewt (subscriber, #32124) [Link]

>> Kernel docs says minimum version is GCC v. 5.1, but it varies depending on architecture.

In coincidental timing, seems that's getting bumped up as of 6.13! This patch is in the pipeline as of recently:

========
From: Ard Biesheuvel <ardb@kernel.org>

Bump the minimum GCC version to 8.1 to gain unconditional support for
referring to the per-task stack cookie using a symbol rather than
relying on the fixed offset of 40 bytes from %GS, which requires
elaborate hacks to support.

Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
========

Building Rust-for-Linux on stable Rust

Posted Sep 24, 2024 20:25 UTC (Tue) by cesarb (subscriber, #6266) [Link] (14 responses)

The issue is that Rust developers are very conservative about new language features; unlike GCC (and clang which copied the behavior from GCC) which always allows the use of language extensions, on Rust all these language features (which would be extensions in GCC) are considered experimental, and only enabled when using the pre-release "nightly" version of the compiler. This allows them to iterate on the feature's design, without having to worry about projects outside the compiler and its standard library depending on them.

> and the kernel always has fallbacks for older GCC and Clang where they're not available.

It doesn't; the kernel uses several non-standard GCC extensions with no fallback, and requires a version of GCC or clang which is recent enough to provide all these required extensions.

Building Rust-for-Linux on stable Rust

Posted Sep 24, 2024 20:50 UTC (Tue) by pizza (subscriber, #46) [Link] (8 responses)

> It doesn't; the kernel uses several non-standard GCC extensions with no fallback, and requires a version of GCC or clang which is recent enough to provide all these required extensions.

While what you are saying is _technically_ correct, the details paint a very different story.

Linux only requires GCC 5.1, which was released *nine years* ago. Versus rustc 1.78, which only landed *four months* ago.

(Or you could use LLVM/Clang intstead of GCC, that requires a ~2.5-year old version (13.0.1)

Source: https://www.kernel.org/doc/html/next/process/changes.html

Building Rust-for-Linux on stable Rust

Posted Sep 24, 2024 22:33 UTC (Tue) by intelfx (subscriber, #130118) [Link] (6 responses)

> While what you are saying is _technically_ correct, the details paint a very different story.

Only if you conveniently ignore some of these details and highlight the others.

> Linux only requires GCC 5.1, which was released *nine years* ago. Versus rustc 1.78, which only landed *four months* ago.

Yes, so how old is the C code in Linux, and how old is the Rust code in Linux? And how much time did it take (since the inception of $LANGUAGE code in Linux) to arrive at this state of affairs?

I’ve said it before, and I’ll say it again: it is unfair and disingenuous that the “new contender” is immediately required to clear the bar that nothing else was subjected to for a damn long time.

Building Rust-for-Linux on stable Rust

Posted Sep 25, 2024 0:06 UTC (Wed) by pizza (subscriber, #46) [Link] (5 responses)

> I’ve said it before, and I’ll say it again: it is unfair and disingenuous that the “new contender” is immediately required to clear the bar that nothing else was subjected to for a damn long time.

It is unfair to deliberately misstate Linux's C compiler requirements as well.

When Linux 5.15 bumped the minimum version to GCC 5.1, the latter had been available for *seven years*. (As opposed to the then-brand-new GCC 11)

Building Rust-for-Linux on stable Rust

Posted Sep 25, 2024 4:48 UTC (Wed) by khim (subscriber, #9252) [Link] (4 responses)

And when Linus presented Linux in comp.os.minix he explicitly have written: It also uses every feature of gcc I could find. And he used the most recent version of gcc he could find.

Now, that doesn't mean that then need to use bleeding version of Rust would stay with us forever, but I don't see why Rust have to be treated differently from how C was treated in the Linux bringup phase.

Building Rust-for-Linux on stable Rust

Posted Sep 25, 2024 11:56 UTC (Wed) by pizza (subscriber, #46) [Link] (3 responses)

> Now, that doesn't mean that then need to use bleeding version of Rust would stay with us forever, but I don't see why Rust have to be treated differently from how C was treated in the Linux bringup phase.

Because unlike Linus's original announcement of Linux 30-odd years ago, Linux is no longer a just-for-fun toy.

If Rust proponents claim that the quality standards/practices of yesteryear are wholly insufficient today then they can't claim "waaah we're being held to a higher standard than ysteryear" when the mirror is turned on them.

The response to the "Rust tooling is still immature and doesn't cover all use cases that some folks need" *fact* should be (and according to the _actual_ R4L devs, is) "We're still working on it; we'll get there", not "waah different standards, give us special treatment instead" or "those use cases don't matter" of the _very_ annoying fanbois that keep coming out of the woodwork.

Building Rust-for-Linux on stable Rust

Posted Sep 25, 2024 14:15 UTC (Wed) by intelfx (subscriber, #130118) [Link]

> The response to the "Rust tooling is still immature and doesn't cover all use cases that some folks need" *fact* should be (and according to the _actual_ R4L devs, is) "We're still working on it; we'll get there", not "waah different standards, give us special treatment instead" or "those use cases don't matter" of the _very_ annoying fanbois that keep coming out of the woodwork.

Now this is a case of shifting the goalposts.

The response is, and always was, "We're still working on it and we'll get there". Nobody has ever said "waah different standards" while asking for "special treatment". The problem with different standards is merely that certain vocal people want "getting there" to happen in zero time. C did not get there in zero time, so why does Rust have to?

Building Rust-for-Linux on stable Rust

Posted Oct 2, 2024 10:57 UTC (Wed) by taladar (subscriber, #68407) [Link] (1 responses)

Seems to me that C is a pretty immature language if you need a 9 year old compiler because some platforms are so reluctant to update that unstable mess to a newer version that they are almost 10 major versions behind the current version.

Seems if you try even a little bit portraying the facts to make the process the other side uses look bad is actually pretty easy.

Building Rust-for-Linux on stable Rust

Posted Oct 2, 2024 12:12 UTC (Wed) by pizza (subscriber, #46) [Link]

> Seems to me that C is a pretty immature language if you need a 9 year old compiler because some platforms are so reluctant to update that unstable mess to a newer version that they are almost 10 major versions behind the current version.

For Linux, said 9-year-old compiler is the *minimum* required version, not the *only* version that is supported/expected to work.

(I am typing these words on a system with the most recent Linux release, compiled using the most recent GCC release)

> Seems if you try even a little bit portraying the facts to make the process the other side uses look bad is actually pretty easy.

Another thing that makes you look bad is deliberately mis-stating documented facts.

Building Rust-for-Linux on stable Rust

Posted Sep 25, 2024 1:22 UTC (Wed) by Wol (subscriber, #4433) [Link]

> Linux only requires GCC 5.1, which was released *nine years* ago. Versus rustc 1.78, which only landed *four months* ago.

istr that for many years Linux had a *maximum* version of gcc, which was positively ancient! If Linux can complain, at different times, that some gccs are too old, while other gccs are too new, then why can't it complain that some rustcs are too old, while others are too new? It is what it is. You just have to use whatever compiler that works.

At the end of the day, the version of the compiler is irrelevant. Seeing as most of the Rust code currently appears to be drivers, if the compiler is contemporaneous with the hardware or feature, what's the problem?

Cheers,
Wol

Building Rust-for-Linux on stable Rust

Posted Sep 25, 2024 2:15 UTC (Wed) by milesrout (subscriber, #126894) [Link] (4 responses)

>are considered experimental, and only enabled when using the pre-release "nightly" version of the compiler. This allows them to iterate on the feature's design, without having to worry about projects outside the compiler and its standard library depending on them.

So why is Linux going to depend on them? How is that supposed to work?

Building Rust-for-Linux on stable Rust

Posted Sep 25, 2024 5:11 UTC (Wed) by admalledd (subscriber, #95347) [Link] (2 responses)

There are a few approaches to this, but the basic one that R4L is (mostly) taking is thus:

1. Recommend a specific known nightly or beta build.
2. Often, the specific nightly build chosen is *actually* the same as the then-just-released "stable compiler", but with the nightly features enabled.

Work-in-progress to get to (2), and really I've used a nightly version from over a year ago (mostly) just fine, the stability-or-not of the total compiler is still (nearly) production worthy, it is the specific features themselves that are more a risk. To mitigate that, Rust-For-Linux *explicitly* tracks and lists exactly which ones they use and why: https://github.com/Rust-for-Linux/linux/issues/2 and that it is (strongly) recommended to not add to that list without good reason.

Further, by linking that github issue as they have, any *changes* in the upstream Rust compiler known to change those unstable features the R4L project depends on gets ping'd to them, which they can then address. Most of the time, the features used by R4L are "user stable" in that "Rust *wants* to enable and let users use it exactly like X, but there are some corner cases that we choose to cause the compiler to error on that should work as well". IE: The features R4L are (mostly) using aren't expected to change much if at all from the developer's perspective. If they are likely to need to change though, due to github bot-magic, they will have plenty of warning and involvement before it even hits a nightly release.

Building Rust-for-Linux on stable Rust

Posted Sep 25, 2024 10:03 UTC (Wed) by sam_c (subscriber, #139836) [Link]

Replying to both you and asahilina: thanks. I'm pleased to hear that there's a strong presumption against adding more such cases especially.

Building Rust-for-Linux on stable Rust

Posted Sep 25, 2024 16:56 UTC (Wed) by ojeda (subscriber, #143370) [Link]

To clarify: we have never recommended nightly or beta compilers (specific or not). We have always supported stable (released) compilers only, until v6.11.

Since v6.11, we support a minimum Rust version, currently 1.78.0. Thus 1.79, 1.80 and 1.81 work too. Beta and nightly compilers should also generally work now.

The Rust project and bindgen have now the kernel in their pre-merge CI. This means that unintentional changes to unstable features in Rust that break Linux do not get merged into Rust.

The #2 issue (and the sublists) definitely helps as you say (and it is one of the reasons why I started it in GitHub, since it cross-references issues), but having the CI is even better. Upstream Rust also has a tag for us, A-rust-for-linux, that we are starting to use.

This has also allowed us to cover the toolchains of a few Linux distributions too.

Please see https://rust-for-linux.com/rust-version-policy, as well as the Kernel Summit talk from LPC a few days ago (the stream/video is not available yet, as far as I understand).

Building Rust-for-Linux on stable Rust

Posted Sep 25, 2024 7:06 UTC (Wed) by asahilina (subscriber, #166071) [Link]

There is a backdoor in the stable compiler to enable nightly features. R4L uses this backdoor, and therefore works with stable Rust builds.

Since the features are still "nightly features" they can and do change, which can break the build with newer compiler versions, but the list keeps getting shorter and we try not to add any new required nightly features unless they are necessary or very important to achieve the R4L goals.

At some point all of the features will be promoted to stable, the backdoor will no longer be necessary, and R4L builds will be guaranteed not to break with newer compilers.

Positive vibes

Posted Sep 24, 2024 16:59 UTC (Tue) by Sesse (subscriber, #53779) [Link] (8 responses)

Is it just me, or does this mood sound much more cooperative than just a few weeks ago?

Positive vibes

Posted Sep 24, 2024 21:20 UTC (Tue) by pbonzini (subscriber, #60935) [Link]

It's not just you, indeed.

Positive vibes

Posted Sep 24, 2024 21:22 UTC (Tue) by rywang014 (subscriber, #167182) [Link] (5 responses)

Yes. I feel the whole fiasco is about "when C breaks Rust, who is responsive to fix". C people stated hard "Rust people are gonna fix them" and in this article we got a nod on this decision. So people can move forward with code.

Positive vibes

Posted Sep 25, 2024 8:40 UTC (Wed) by pbonzini (subscriber, #60935) [Link] (4 responses)

At the same time there's also more willingness to collaborate than what transpired from the infamous "you're not going to make us learn Rust" quote.

Positive vibes

Posted Sep 25, 2024 12:05 UTC (Wed) by pizza (subscriber, #46) [Link] (3 responses)

> At the same time there's also more willingness to collaborate than what transpired from the infamous "you're not going to make us learn Rust" quote.

FWIW, sampling two points from from a locally-noisy source does not make for good generalizations.

Positive vibes

Posted Sep 25, 2024 12:16 UTC (Wed) by Wol (subscriber, #4433) [Link]

The problem is, the squeaky wheel gets all the oil.

Cheers,
Wol

Positive vibes

Posted Sep 26, 2024 19:29 UTC (Thu) by pbonzini (subscriber, #60935) [Link] (1 responses)

Well, if Ted went from expressing frustration to:

> There is a need for documentation and tutorials on how to write filesystem code in idiomatic Rust. He said that he has a lot to learn; he is willing to do that, but needs help on what to learn.

Then it's a lot more than just "two data points".

Positive vibes

Posted Sep 26, 2024 20:06 UTC (Thu) by Wol (subscriber, #4433) [Link]

> > There is a need for documentation and tutorials on how to write filesystem code in idiomatic Rust.

I'm pleased to see he wrote "documentation AND TUTORIALS".

Never forget, without tutorials, documentation is usually written in double dutch ... (says he who's just been reading a load of documentation and wondering what the hell it's going on about ...)

Cheers,
Wol

Positive vibes

Posted Oct 1, 2024 22:04 UTC (Tue) by dmv (subscriber, #168800) [Link]

Makes me wonder how many behind-the-scenes conversations were had to arrive at the positivity.

As an onlooker, the basic idea that the kernel is developed out in the open is an attractive notion, but it’d be really interesting to have the curtains pulled back a bit to see how much actually is somewhat behind-closed-doors/out of the public eye. I would assume some of it would have to be. Given that companies have kernel developers on staff, some of that stuff is gonna be company-internal, of course. But I’m interested less in that part of it. The kernel development project is its own massive social experiment of sorts. Surely human-sensitive things are easier to manage out of the public eye. What else?

Unstable compilers

Posted Sep 24, 2024 18:39 UTC (Tue) by sam_c (subscriber, #139836) [Link] (13 responses)

> Airlie said that it is too soon to ask the Rust community for a completely stable compiler to build kernel code with; there just is not yet enough Rust code in the kernel to make that happen.

I don't understand this, this feels like it ought to be a prerequisite. If the language isn't ready yet with the pace at which it moves and the lack of a LTS release compiler, perhaps it isn't ready for the kernel yet either.

Unstable compilers

Posted Sep 24, 2024 21:44 UTC (Tue) by Baughn (subscriber, #124425) [Link] (11 responses)

This describes C, for most of the history of thethe kernel.

Unstable compilers

Posted Sep 25, 2024 1:02 UTC (Wed) by sam_c (subscriber, #139836) [Link] (10 responses)

Please elaborate. I am unfamiliar with the C you describe.

Unstable compilers

Posted Sep 25, 2024 5:16 UTC (Wed) by admalledd (subscriber, #95347) [Link] (9 responses)

TL;DR: Until "recent history" of about as recent as ten years ago, compiling the Linux Kernel had a very odd list of "these are the only compiler versions *known* to work correctly, try any others and they are between you and the mantissa of your FPU doing a DIV 0". Such as specific patch levels of GCC or CLANG.

You could (especially as you approach the modern era) still often *compile* with whatever GCC (and sometimes CLANG) version so long as it was above some horribly old bare minimum, but it was not uncommon to have subtle miscompilations in more esoteric drivers, or flat out some drivers not compiling, etc.

Many forget this, because distros would baseline on these known-good compiler versions, so the majority of users would never know, often even kernel developers.

Unstable compilers

Posted Sep 25, 2024 9:32 UTC (Wed) by sam_c (subscriber, #139836) [Link] (6 responses)

Thanks. I guess I'd make the distinction between new features only available in and bugginess though.

Unstable compilers

Posted Sep 25, 2024 9:32 UTC (Wed) by sam_c (subscriber, #139836) [Link]

I meant to add: ... because you can more easily backport bug fixes.

Unstable compilers

Posted Sep 25, 2024 9:46 UTC (Wed) by Wol (subscriber, #4433) [Link] (4 responses)

> I guess I'd make the distinction between new features only available in and bugginess though.

You need to add to that list "new features either enabled by default, or often enabled by force, that break the kernel".

One only has to look at all the complaints about UB, where gcc was optimising code away. Quite often (a) this stuff was NEEDED, and (b) there was no flag to disable the optimisation responsible for removing them.

So the compiler wasn't buggy, it was working as designed. And if the die-hards are going to complain and say "well Rust needs to call C to access the hardware", it was not at all unusual for C to have to call assembler to access the hardware, because the C compiler was just deleting the code as UB with no way to stop it. Hence my comments in various places about mathematics is not reality.

The difference is, it appears the Rust devs seem much more amenable to treating "you're optimising away necessary code" as a bug than the C folks were.

Cheers,
Wol

Unstable compilers

Posted Sep 25, 2024 9:52 UTC (Wed) by pbonzini (subscriber, #60935) [Link] (3 responses)

> it appears the Rust devs seem much more amenable to treating "you're optimising away necessary code" as a bug than the C folks were.

The optimization backend in the end is the same, the difference is that the language itself is designed to make it harder to trigger undefined behavior.

For example the level of aliasing is declared explicitly by separating shared and exclusive references (& and &mut). Developers can only turn a shared reference into an exclusive one via UnsafeCell (and then the code needs to be explicitly marked as unsafe) or wrappers thereof (which are carefully designed to avoid undefined behavior).

Unstable compilers

Posted Sep 25, 2024 10:42 UTC (Wed) by Wol (subscriber, #4433) [Link] (2 responses)

> > it appears the Rust devs seem much more amenable to treating "you're optimising away necessary code" as a bug than the C folks were.

> The optimization backend in the end is the same, the difference is that the language itself is designed to make it harder to trigger undefined behavior.

Maybe that's the effect. But even today the C devs seem keen on creating new undefined behaviours. In Rust, undefined behaviour is viewed as a bug in the language (or as "you can only do that in an unsafe block") as far as I can tell. Completely diametric views on UB.

Cheers,
Wol

Unstable compilers

Posted Sep 25, 2024 18:10 UTC (Wed) by sunshowers (guest, #170655) [Link]

Yes, that is correct, and Rust is better for it. Under the assumption that unsafe code is correct, safe code doesn't have UB.

I would rather optimization 'alpha' be extracted through principled approaches like enabling the use of "restrict" on &mut pointers.

Unstable compilers

Posted Sep 26, 2024 20:04 UTC (Thu) by ralfj (subscriber, #172874) [Link]

> In Rust, undefined behaviour is viewed as a bug in the language (or as "you can only do that in an unsafe block") as far as I can tell. Completely diametric views on UB.

Yes -- mostly the latter. I think Undefined Behavior is a great tool for language designers; they can use it to enable powerusers to get the compiler to do things you'd never (well, not practically) get it to do without. But it is the language designer's responsibility to yield this tool with care.

A more lengthy discussion of this point is in my blog: https://www.ralfj.de/blog/2021/11/18/ub-good-idea.html

In C, things can be UB either because the standard says they are UB, or because the standard is entirely silent about that case. The latter is considered a spec bug in Rust, it is not acceptable for us to have things be "implicitly" UB. The former still happens, but we are doing our best to describe those cases as unambiguously as possible. Rust doesn't have a spec yet, so we still have a lot of work ahead of us, but Miri [1] helps a lot: if we understand our UB rules well enough to wire them into a tool that can automatically test for UB, that's a great first step towards an unambiguous specification! I think something like Miri would be nearly impossible to do for C, since the standard is just not sufficiently clear about many corner cases.

[1]: https://github.com/rust-lang/miri/

Unstable compilers

Posted Sep 25, 2024 17:56 UTC (Wed) by ballombe (subscriber, #9523) [Link] (1 responses)

> You could (especially as you approach the modern era) still often *compile* with whatever GCC (and sometimes CLANG) version so long as it was above some horribly old bare minimum, but it was not uncommon to have subtle miscompilations in more esoteric drivers, or flat out some drivers not compiling, etc.

Because all rust compilers are guaranteed to be bug-free ?

Unstable compilers

Posted Sep 25, 2024 18:29 UTC (Wed) by admalledd (subscriber, #95347) [Link]

Certainly not, but the stability of `rustc` is often much higher than many give it credit for, and that Rust gives much better ABI(*)/linking guarantees. It is often not plausible to use a different GCC version from what compiled the kernel to compile custom modules for example. Rust can provide much stronger guarantees here by default, though as mentioned in the article there is in-kernel work to be done for the module symbol naming/linking.

Rust in *most* cases will either halt compilation with an Error, and InternalCompilerError, or result in symbols that will fail at linking time instead of (used to be more) commonly "compiling" fine but crashing/failing at runtime or `insmod` time.

* Rust technically has no ABI, but you can expose/use one via various exports/macro things, commonly of course a "c-abi". This is mostly handled by `rust-bindgen` for automation with a dash of human control when required.

Unstable compilers

Posted Sep 25, 2024 12:47 UTC (Wed) by tialaramex (subscriber, #21167) [Link]

Take for example 128-bit integer support. Rust has 128-bit integers. Code to handle these e.g. formatting them for output as text - exists but in a typical Rust application which doesn't actually have any 128-bit integers the code for them never ends up in the resulting binary.

But, Linux currently just says nope, we'll take everything just in case, because a loadable module might use it. A few kilobytes of formatting for types we never use? Give me those too.

So for Stable Rust Linux would get a bunch of 128-bit integer formatting code it has no use for, on the off chance that somebody might some day write a loadable module which expects 128-bit integer formatting to work. In Rust for Linux they've instead just made this code conditional and switched off the condition, it won't compile in your loadable module and the kernel avoids a few kilobytes of "format 128-bit integers" code.

The push back from the Rust community is they're not interested in doing a bunch of extra work to help people who want all the symbols but then wish somebody else would do the work to slim it down for them, pick a lane. So the Rust for Linux change will never land in the stable Rust in this form.