[go: up one dir, main page]

|
|
Log in / Subscribe / Register

Committing to Rust for kernel code

Committing to Rust for kernel code

Posted Nov 23, 2023 0:25 UTC (Thu) by Conan_Kudo (subscriber, #103240)
In reply to: Committing to Rust for kernel code by ojeda
Parent article: Committing to Rust for kernel code

>> Having the kernel build break every six weeks is basically madness
>
> What do you mean by "break"?

It fails to compile because something has broken in rustc in some way, or the unstable feature has changed and broken something, or both. It happens without fail.

>> the lack of priority to eliminate usage of nightly/unstable Rust features
>
> This is false. It is, in fact, a priority for us: https://rust-for-linux.com/unstable-features.

You still allow *new* unstable features to be used. That list will keep growing. And in fact, I know it will once the DRM stuff is incorporated into mainline RFL (whenever that happens). It's only a priority once you start saying "no" to adding more of them. That can't happen for a while yet since the various subsystems don't yet have Rust abstractions in mainline.

>> lack of a proper specification
>
> I am not sure what you mean by this (and other points you make in this second paragraph), but if you are talking about a Rust language specification, then that is not really on us. In any case, the Rust project has decided to start working on an official specification for the Rust language. Ferrocene has one too.

I mean that you do not have a specification of how you want to handle Rust *in* Linux itself. This makes it really hard for people to figure out how to grade it and grok it when patches are being submitted to build new abstractions or kernel modules.

>> the lack of interest from kernel developers to review the Rust abstraction patches needed to bring in Rust-based drivers
>
> Different subsystems have different timelines, requirements, bandwidth and so on. You cannot expect every kernel maintainer to be on board or have time to work on Rust this early on. Having said that, as I mentioned in the summit, during this last year we have seen an increased interest from kernel maintainers, and in fact some of them have already been very helpful and supportive.

No, that's true. But I *can* expect the ones that *are* already doing Rust in other places to do more than nothing in the kernel. *My* patch load keeps growing because *nothing* has made it upstream in a year. It's extremely frustrating.

>> I do not think enabling Rust in a distribution kernel would be reasonable to consider for a few years.
>
> A few major distributions told me it should be possible for them to support Rust for Linux in the future. Ubuntu, for instance, provides packages with the Rust toolchain needed for the kernel.

I am shocked. I was also shocked when you said you reached out to all the major distributions, because I have not seen you reach out to Fedora *at all*.

Insofar as Ubuntu shipping it, it's pretty easy when there's nothing to ship.

The Fedora Asahi Remix kernel has one of the most significant Rust-based kernel modules (the Asahi DRM driver), and it can't even build on Rust 1.73 because something is broken in rustc itself (though nobody cares because it's broken somewhere related to a nightly/unstable feature). My next rebase will be the next time I try with Rust 1.74, and I'm dreading it.

I have recommended to upstream Fedora to *not* enable it because it's so risky to enable right now. Since we track upstream Rust on their compilers, Linux Rust code is busted all the time.


to post comments

Committing to Rust for kernel code

Posted Nov 24, 2023 2:33 UTC (Fri) by ojeda (subscriber, #143370) [Link] (8 responses)

> It fails to compile because something has broken in rustc in some way, or the unstable feature has changed and broken something, or both. It happens without fail.

The kernel only supports a single version of rustc for the time being (as the page mentioned above explains). Obviously, if you are using an unexpected compiler and also have out-of-tree code with extra unstable features, then you will need to handle the upgrade work yourself (which is why upstreaming would help).

One of the reasons why we started tracking the latest Rust release more closely is because it could help some very early users, including you. So I am not sure what you still expect us to do — I already told you back then that the kernel tracking the very latest Rust release would not solve all your problems (and, in fact, if you are dealing with a ton of out-of-tree code, then it may have been best for you that the kernel did not change versions that often and simply provide the right toolchain on your side).

> You still allow *new* unstable features to be used. That list will keep growing. And in fact, I know it will once the DRM stuff is incorporated into mainline RFL (whenever that happens). It's only a priority once you start saying "no" to adding more of them. That can't happen for a while yet since the various subsystems don't yet have Rust abstractions in mainline.

Please read the second page I mentioned earlier. You seem to believe that the features included in the initial minimal Rust merge were the only ones that we would need to use, forever. This is not the case, and we even have an issue (started in 2020) in our tracker listing the features we expect to need (linked from the other page too).

Moreover, it is also not the case that all features are treated equally (e.g. not all have the same timelines or the same degree of stability) nor that we add them just because we fancy them (the default answer is "no", in fact), which is what one could understand from your different comments in this article. Again, please refer to the page above.

Furthermore, whether some code is in mainline or not is irrelevant. You seem to believe that any maintainer in the kernel is able to start using any feature they like, which is not true. On top of that, for code outside abstractions, we are way stricter on the use of unstable features.

Finally, we are doing our best to work with the Rust project (and other entities) to try to get to the point where we can eventually declare a minimum version. So I really do not appreciate that you imply we are somehow careless about our usage of unstable features.

> I mean that you do not have a specification of how you want to handle Rust *in* Linux itself. This makes it really hard for people to figure out how to grade it and grok it when patches are being submitted to build new abstractions or kernel modules.

We are happy to help here, but it is unclear what you mean by "specification". What would you expect to see there? If you mean a contribution guide, we already have one among other documentation. Of course, we are still improving it and completing it, but others have already found it useful and successfully submitted patches.

> But I *can* expect the ones that *are* already doing Rust in other places to do more than nothing in the kernel. *My* patch load keeps growing because *nothing* has made it upstream in a year. It's extremely frustrating.

Why would you expect that? I am confused. Someone may like Rust, and even use it elsewhere (outside the kernel or in other areas of the kernel), but that does not mean they agree with using it within the kernel or in a particular area of it, or that they have the bandwidth to help you, or the expertise in a particular topic, or they may simply only work in the kernel as part of their job which may not include time for that, etc.

Anyway, I understand it can be frustrating, but it is what it is. Kernel maintainers, even those willing to help you, cannot simply merge code for the only reason of reducing your patch load. And, in case that is part of the confusion, kernel maintainers cannot randomly merge changes for areas they do not maintain. (To be clear, I don't know who you are referring to, i.e. I am not talking about anybody in particular.)

> I am shocked. I was also shocked when you said you reached out to all the major distributions, because I have not seen you reach out to Fedora *at all*.

Fedora already tracks the latest Rust compiler and Josh Stone has been involved in our issue tracker discussing the versioning policy (just like you were — see the first point).

And to be clear, I reached out to some that seemed to be able to provide a recent enough Rust release in their rolling user-facing releases. Something like Debian stable will obviously take longer, which is why when it came up in the summit I suggested providing the toolchain at kernel.org as another alternative that could perhaps help some people. Distributions could also provide a special package, like Ubuntu is doing — that would be great too and we would appreciate it.

> Insofar as Ubuntu shipping it, it's pretty easy when there's nothing to ship.

The point was that Ubuntu provides the toolchain required by the kernel, thus compiling the kernel should always work for them regardless of how much code or unstable features are in use in mainline.

> The Fedora Asahi Remix kernel has one of the most significant Rust-based kernel modules (the Asahi DRM driver), and it can't even build on Rust 1.73 because something is broken in rustc itself (though nobody cares because it's broken somewhere related to a nightly/unstable feature). My next rebase will be the next time I try with Rust 1.74, and I'm dreading it.

We cannot promise out-of-tree code compiles. The best route is to get the code upstreamed and use the compiler version expected by the kernel. The DRM maintainers are supportive of Rust, so that is definitely an advantage in your case.

I don't know why you say "nobody cares" or why you claim "something is broken in rustc itself", though. Please see the points above.

> I have recommended to upstream Fedora to *not* enable it because it's so risky to enable right now.

If by "risky" you mean "I get compile errors because I am carrying out-of-tree code with extra unstable features and use an unexpected compiler on top", sure. In any case, I would also not enable it until there is an actual in-tree module that their users actually need and the support is deemed production-ready. So for the time being I agree with your recommendation, even if for different reasons.

> Since we track upstream Rust on their compilers, Linux Rust code is busted all the time.

Linux Rust code is definitely not "busted". Again, if you are using out-of-tree code and an unexpected compiler on top of that, then that is something you will need to handle yourself.

Sincerely, it sounds to me like you are doing some things on your own, facing some difficulties for some time and then deciding that the best approach is to complain here. We already tried to help you in the past, and it seems you did not notice we started tracking the latest release. I think a better approach would be reaching out to us to see if we can help you further or at least to understand each other's context/constraints better.

Committing to Rust for kernel code

Posted Nov 24, 2023 12:52 UTC (Fri) by Conan_Kudo (subscriber, #103240) [Link] (5 responses)

Most of the problems are in the forked "alloc" crate that's in-tree, not in the Asahi DRM driver code. The website also says not to send patches for making the tree compatible with newer Rust compilers, so we don't. Instead, we have patches downstream in the Asahi tree to make adjustments to fix it for newer Rust versions.

My complaints around my patch load are centered around two things: you don't want fixes for newer Rust versions *and* DRM (despite being mostly enthusiastic about Rust) has been stalling on the reviews. The latter issue is apparently resolving itself now, so we may see progress on that front. But I'm still not happy about the fact you do not accept patches for fixing the code to work on newer Rust versions.

Committing to Rust for kernel code

Posted Nov 25, 2023 16:58 UTC (Sat) by ojeda (subscriber, #143370) [Link] (4 responses)

> Most of the problems are in the forked "alloc" crate that's in-tree, not in the Asahi DRM driver code.

Glad to hear that; then if you take the upgrades we do, most of your problems should go away.

> The website also says not to send patches for making the tree compatible with newer Rust compilers, so we don't. Instead, we have patches downstream in the Asahi tree to make adjustments to fix it for newer Rust versions. (...) you don't want fixes for newer Rust versions

The website asks not to send upgrades to newer versions because we are handling those on our side already. What we recommend is that users take the upgrades that we put upstream, and then you do not need to deal with alloc anymore.

> you don't want fixes for newer Rust versions (...) But I'm still not happy about the fact you do not accept patches for fixing the code to work on newer Rust versions.

Again, we support a single compiler version for the moment, and we are already upgrading to newer versions if there are no blockers. Thus there are no "fixes" needed (modulo mistakes in the upgrade, of course).

I suspect you may be talking about supporting multiple versions and wanting to add workarounds for that, but that is a different discussion/approach altogether, and I don't recall you suggesting that? In fact, in that GitHub issue, you said:

"I think that as long as you use unstable features, you should be tracking the latest Rust compiler at their lifecycle. Once you're not using unstable features anymore, then widening the range of compilers supported is a reasonable conversation to have."

Committing to Rust for kernel code

Posted Nov 25, 2023 17:21 UTC (Sat) by Conan_Kudo (subscriber, #103240) [Link] (3 responses)

The approach we're currently taking is to do the minimal effort to enable the Rust compiler version shipped in Fedora, which is nearly always ahead of what Rust-for-Linux is tracking. Keep in mind that we're shipping new versions of rustc shortly after upstream releases them. I don't particularly care about supporting multiple versions of Rust, but depending on what the change is, it may look like it might work for multiple Rust versions even if that's completely untested.

Committing to Rust for kernel code

Posted Nov 27, 2023 15:48 UTC (Mon) by ojeda (subscriber, #143370) [Link] (2 responses)

I am aware of all that, which is precisely why I wrote what I wrote.

I am still not understanding what you are trying to suggest. That we take "completely untested" patches that attempt to support multiple versions? That is not how things work.

And even if we decided to support multiple versions, that would still not solve your problems, since you would still be ahead.

Committing to Rust for kernel code

Posted Nov 27, 2023 16:10 UTC (Mon) by Conan_Kudo (subscriber, #103240) [Link] (1 responses)

We are doing that approach downstream because we need to. Obviously for upstream purposes, we'd do things differently. Our downstream patches are intended to enable us to build with the latest Rust compiler available in Fedora. Your approach with RfL is mostly fine, it's just we're not at a place where I would be comfortable recommending it to be enabled in a mainline distro kernel. The forked alloc crate issue has to be resolved first.

Committing to Rust for kernel code

Posted Nov 27, 2023 16:26 UTC (Mon) by ojeda (subscriber, #143370) [Link]

> Your approach with RfL is mostly fine,

As I told you both here and in Zulip, please explain what is your proposal for improvement. If you do not have one, please do not assume or imply that things could be done better.

The Rust versions we pick may not be the best for you (in fact, they are already the best ones we can pick for you), but that does not mean Rust for Linux is governed or developed in a bad way given the current constraints.

> it's just we're not at a place where I would be comfortable recommending it to be enabled in a mainline distro kernel.

Like I told you in Zulip, I am not sure why this is relevant. Whatever you recommend does not change our constraints.

I also already told you that we ourselves tell general purpose distributions that it is early to enable the upstream Rust support.

> The forked alloc crate issue has to be resolved first.

No, the reason is not alloc. The fact that you say so concerns me, since that is an implementation detail for users.

The actual reasons are, for instance, that there are no users in-tree, MODVERSIONS, etc.

Committing to Rust for kernel code

Posted Nov 25, 2023 8:02 UTC (Sat) by marcan (guest, #103032) [Link] (1 responses)

Hey Miguel,

I just opened a thread on the Zulip to talk about this. Conan doesn't have too much visibility into the actual work going into our downstream branch Rust support, but I do. And indeed, the biggest problem is 100% upstream: it's the `alloc` crate. That's what is breaking with almost every compiler version bump, and I haven't seen any discussion or plans for how that will be resolved yet.

Upstreaming, the small handful of features used in the other crates, etc. are all things with solid plans going forward, and I look forward to the future for Rust for Linux. But as long as `alloc` is using 100+ unstable features directly upstream, all the work to eventually drop the dozen features used in the `kernel` crate is rather useless. Being able to unpin the Rust compiler version is very important for distros to start taking this seriously and reduce friction enabling Rust support, so I hope there is some plan to eventually solve this. Maybe there is and I just haven't found it? Would be happy to hear your thoughts :)

Committing to Rust for kernel code

Posted Nov 25, 2023 15:44 UTC (Sat) by ojeda (subscriber, #143370) [Link]

The alloc topic has been in discussion since essentially the beginning of the project. The original plan discussed with upstream Rust (and others) is documented in-tree, please see the rust/alloc/README.md file. In short, the alloc fork is meant to be temporary and the discussions with upstream Rust are ongoing.

Committing to Rust for kernel code

Posted Nov 24, 2023 11:38 UTC (Fri) by khim (subscriber, #9252) [Link]

> though nobody cares because it's broken somewhere related to a nightly/unstable feature

Also, please, note that this is gross exaggeration: while unstable features can be deliberately changed at any time (that's why they are marked “unstable”, after all) if something is broken by accident then rustc developers take bug-reports seriously.

Have you tried to create in issue against rustc? Yes, they may ask you to change your code, if you use unstable feature, but “nobody cares” is the exact opposite from what I observed.

Firefox and many other large companies use Rust unstable features — do you think it would have worked for them if nightly features were in the state of being perpetually broken?

And if you have filed the issue then looking on it would be better than talking about abstract issues in vacuum.


Copyright © 2026, Eklektix, Inc.
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds