Committing to Rust for kernel code

By Jonathan Corbet
November 22, 2023

Rust has been a prominent topic at the Kernel Maintainers Summit for the last couple of years, and the 2023 meeting continued that tradition. As Rust-for-Linux developer Miguel Ojeda noted at the beginning of the session dedicated to the topic, the level of interest in using Rust for kernel development has increased significantly over the last year. But Rust was explicitly added to Linux as an experiment; is the kernel community now ready to say that the experiment has succeeded?

The Rust-for-Linux project has added a full-time engineer in the last year, Ojeda said, and a student developer as well. Various companies have joined in to support this work. There is also work underway to get the Coccinelle tool working with Rust code. A priority at the moment is bringing in more reviewers for the code that is being posted.

On the toolchain front, work on gccrs, the GCC-based Rust compiler, has slowed significantly. The GCC code generator for rustc is showing better progress; it can compile kernel code now and has been merged into the compiler. This GCC-based backend will enable the expansion of Rust support to architectures that are not supported by the LLVM-based rustc. Meanwhile, the Rust project itself is increasing its involvement in this work; this is good, since the kernel has some unique requirements and will need guarantees that language changes won't break kernel code in the future.

Within the kernel, work is proceeding in a number of subsystems. The Rust implementation of Android's binder is working well and its performance is on a par with the C implementation. The amount of unsafe code that was needed to get there was pleasingly small. Filesystem bindings are the subject of work by Wedson Almeida Filho, who is targeting read-only support for now. The object there is to make it possible to implement a filesystem in 100% safe Rust.

In general, he is finding an increasing number of maintainers who are open to the idea of using Rust. That leads to an issue the Rust developers have run up against, though. It would be good to have some reference drivers in the kernel as an example of how drivers can be written and to make it possible to compare Rust and C drivers. The best way to do that often seems to be to merge a Rust driver that duplicates the functionality of an existing C driver — but that sort of duplicate functionality is not welcomed by maintainers. Perhaps, he said, it would be good to allow a few duplicate drivers that are not meant for actual use, but only as examples for other developers to use.

There are some other challenges; upstreaming the block-layer abstractions has run into some resistance. Virtual filesystem layer maintainer Christian Brauner said that he is not opposed to merging those abstractions, but he would rather not do that and see filesystems built on it right away. He would prefer to see an implementation of something relatively simple, along the lines of the binder driver, to show that things work as expected.

A driver soon?

Dave Airlie, the maintainer of the DRM (graphics) subsystem, said that, if he has his way, there will be a Rust DRM driver merged within the next couple of releases. Christoph Hellwig shot back that Airlie was willing to "make everybody's life hell" so that he could play with his favorite toy. Merging Rust, Hellwig said, would force others to deal with a second language, new toolchains, and "wrappers with weird semantics". Dan Williams said that the current situation "is what success looks like", and that the kernel community was already committed to Rust.

Airlie continued that a lot of the Rust work is currently blocked in a sort of chicken-and-egg problem. Abstractions cannot be merged until there is a user for them, but the code needing those abstractions is blocked waiting for code to land in multiple subsystems. As a result, developers working on Rust are dragging around large stacks of patches that they need to get their code to work. Breaking that roadblock will require letting in some abstractions without immediate users. Ojeda agreed that this problem has been slowing progress, but said he has tried not to put pressure on maintainers to merge code quickly. In the case of networking, ironically, the Rust developers had to ask the networking maintainers to slow down merging Rust code.

The conversation took several directions from there. Greg Kroah-Hartman said that merging the binder driver would be a good next step; it is self-contained, has a single user that is committed to its maintenance, and doesn't touch the rest of the kernel. Kees Cook disputed the description of Rust as a "toy", saying that there is a lot of pressure to not use C for new code; Hellwig responded that the developers would have to rewrite everything in Rust, otherwise the resulting dual-language code base would be worse than what exists now.

Dave Chinner worried that maintainers lack the expertise to properly review the abstractions that are being merged. Airlie replied that maintainers merge a lot of C APIs now without really understanding how they work. A lot of mistakes have been made in the process, but "we're still here". When things turn out to be broken, they can be fixed, and that will happen more quickly if the code goes upstream.

Ted Ts'o expressed concern about the burden that adding Rust code will place on maintainers. The Rust developers are setting higher standards than have been set in the past, he said. Getting good abstractions merged is one thing, but who is responsible for reviewing drivers, and how will tree-wide changes be handled? The Rust effort, he said, is getting to a point where it is impacting a growing part of the community.

Williams pointed out that the previous session had discussed how hard it is to get kernel subsystems to move to new APIs; now, he said, there is talk of moving to a whole new language. Hellwig said that the real problem is that the Rust bindings tend to work differently than the C APIs they provide abstractions for; the new APIs may well be better, but they are still completely new APIs. What should be done, he said, is to first fix the C APIs so that they are directly usable by Rust code. He proposed that, for each subsystem that is considering bringing in Rust code, a year or two should first be spent on cleaning up its APIs along those lines. Ojeda said that this kind of API improvement has already happened in some subsystems.

Linus Torvalds said that he was seeing a divide between the filesystem and driver maintainers. Developers on the filesystem side tend to be more conservative, while the driver world "is the wild west". Driver authors tend not to understand concurrency, he said, and a lot of the code there is broken and unfixable. So it is unsurprising that there is interest in bringing in a language that better supports the writing of correct and safe code.

Brauner said that Rust can help with a lot of problems, since the compiler can keep a lot of bugs from making it into the kernel. But he worried about whether there would be maintainer and development support for it a few years from now. Airlie again mentioned developers with out-of-tree code needed by Rust code; Cook answered that the people shepherding that code are maintainers, and that bringing it in would bring the maintainers with it. Airlie added that those maintainers are the sort of younger developers that the kernel community would like to attract.

Chinner said that he would like to see a reimplementation of the ext2 filesystem in Rust. It is a complete filesystem that makes wide use of the kernel's APIs, but it is still small enough to read and understand. If the Rust APIs can support an ext2 implementation, they will be enough to implement others as well. Meanwhile, the ext2 implementation would be good reference for maintainers, who could compare it to the C version.

Confidence

Ts'o asked when the community would feel enough confidence that it could have modules where the only implementation is in Rust. Binder could be a good start, he said, perhaps followed by a driver that sees wider use. Airlie said that he is considering a virtual graphics driver that reimplements an existing C driver. There is also the driver for Apple M1 GPUs. He is feeling a fair amount of pressure to get it upstream and is wondering if there is any reason why he should keep it out. After that, he would love to see a rewrite of the Nouveau driver for NVIDIA GPUs.

Arnd Bergmann said those drivers could be OK, but that it will be quite a bit longer before something like a keyboard driver could be merged; the toolchain just isn't ready, he said, for a driver what would be widely used. That led to a question about the frequent version upgrades being seen in the kernel, which moved to Rust 1.73.0 for 6.7. That upgrade process will eventually stop and a minimum Rust version will be set once the important features that the kernel depends on have stabilized. He said that he has been working to get the kernel code into the Rust continuous-integration tests to help ensure that it continues working as the compiler and language evolve.

Bergmann said that he didn't plan to look seriously at the language until it could be compiled with GCC. Torvalds answered that, while he used to find problems in the LLVM Clang compiler, now he's more likely to find problems with GCC instead; he now builds with Clang. Ojeda said that he is working on finding developer resources for gccrs; the project is currently sitting on over 800 out-of-tree patches and still has a lot of work to do on top of that. GCC support will be a while, he said.

Ts'o complained that the language still isn't entirely stable. This could be a particular problem for the confidential-computing community; they are concerned about security and, as a consequence, about backports to long-term-support kernels. But if those kernels are on different Rust versions, those backports will be problematic. Ojeda said that, while it is a "crazy idea", backporting the version upgrades could be considered. He doesn't think that the change rate will be high enough to be a problem, though.

At the conclusion, Torvalds pointed out that there have been problems over the years with GCC changes breaking the kernel; the same will surely happen with Rust, but it will be the same thing in the end. The session, well over time, was brought to a halt at this point. Whether the kernel community has truly concluded that it is committed to Rust remains to be seen; there will almost certainly be pull requests adding significant Rust code in the near future.

Index entries for this article
Kernel	Development tools/Rust
Conference	Kernel Maintainers Summit/2023

Committing to Rust for kernel code

Posted Nov 22, 2023 15:53 UTC (Wed) by tux3 (subscriber, #101245) [Link] (1 responses)

This level of tension between maintainers who aren't too thrilled to bear the extra cost and others who deem it worth it, doesn't seem like it is going away on its own.

Has there been much discussion about possible forms of analgesia that I might have missed? Perhaps people willing to answer questions, resources, or other commitments from the Rust side that may help?

Committing to Rust for kernel code

Posted Nov 22, 2023 16:01 UTC (Wed) by corbet (editor, #1) [Link]

The Rust-for-Linux folks have been clear from the beginning that they understand the costs and that they plan to be there to help maintainers through this process.

Committing to Rust for kernel code

Posted Nov 22, 2023 16:22 UTC (Wed) by willy (subscriber, #9762) [Link] (3 responses)

Agreed with Dave that ext2-in-rust would be a very worthwhile project. I've reviewed the most recent batch of VFS abstractions and they suffer from "we're just implementing toy filesystems" syndrome.

That's not to say that I want anyone to disappear for a year and come back with rext2. I'd like to see somebody working on it publicly with regular postings about what they're finding.

In particular, I'd like to see such a filesystem either following the existing C s_op / f_op / i_op / a_ops abstractions, or an explanation of why they are bad and a proposal for what a better set of methods would be. I don't think we've necessarily got the perfect abstractions, but they are here and work well enough that we have ~80 filesystems. This is a great chance to fix them up if there's a better model.

Also, rext2 shouldn't be a "port" of C ext2. It should be a reimplementation of the filesystem, using modern infrastructure (ie not buffer heads).

Committing to Rust for kernel code

Posted Nov 22, 2023 22:50 UTC (Wed) by amiculas (guest, #167443) [Link] (2 responses)

Just so I can be less ignorant, what's the modern alternative to buffer heads?

Committing to Rust for kernel code

Posted Nov 22, 2023 22:55 UTC (Wed) by willy (subscriber, #9762) [Link] (1 responses)

Usually, it's iomap. Ritesh has done an excellent patch set this week that uses iomap for data. For directories, we're not really sure what the right abstraction is -- should we use the page cache raw; should we have a thinner version of buffer heads; should we use XFS buffer cache; should we use iomap? All TBD.

Committing to Rust for kernel code

Posted Dec 10, 2023 11:26 UTC (Sun) by rincebrain (subscriber, #69638) [Link]

...except almost all of iomap is EXPORT_SYMBOL_GPL, so anyone working on things that aren't GPL can't play in that sandbox.

Committing to Rust for kernel code

Posted Nov 22, 2023 16:39 UTC (Wed) by b7j0c (guest, #27559) [Link] (63 responses)

Even though I am enjoying coding in Rust myself and advocate it to others, I still feel like I am in the uninformed minority that is against introducing a major tool-based segmentation into the kernel. Given that the C will always be part of the kernel, isn't the best-case scenario here splitting kernel dev into "Rust side" and "C side"? Maybe some better informed folks here can clarify...

Committing to Rust for kernel code

Posted Nov 22, 2023 18:26 UTC (Wed) by pizza (subscriber, #46) [Link] (57 responses)

> Given that the C will always be part of the kernel,

A primary goal of this Rust-in-Linux effort is to change that via a kernel-of-thesius approach.

First make it so that, if you have a certian common platform (Android) or hardware (Apple), you need to use Rust. Then you incrementally push it it deeper into core layers, effectively making it a requirement for every other platform. Once it's required period, then incrementally replace or drop the rest (obsolete filesystems, device drivers, etc) until it's all Rust.

So they may or may not be explicitly stating it, but making Linux Rust-First is the only logical conclusion.

(Now if I want to go more conspiratorly-minded, once can also consider the Rust-on-Linux efforts to be an extension of the decades-long effort to purge dependencies on copyleft tooling, notably GCC. Linux is copyleft in name only, due to an utter lack of enforcement)

Committing to Rust for kernel code

Posted Nov 22, 2023 18:51 UTC (Wed) by Wol (subscriber, #4433) [Link] (27 responses)

> (Now if I want to go more conspiratorly-minded, once can also consider the Rust-on-Linux efforts to be an extension of the decades-long effort to purge dependencies on copyleft tooling, notably GCC. Linux is copyleft in name only, due to an utter lack of enforcement)

In which case, wouldn't it make sense for all kernel Rust code to be liberally licenced?

I'm thinking about the same thing with Scarlet. Not that I probably will, but re-writing it in Rust might be an idea, and requiring Rust code to be MPL, or Apache, or MIT, or something like that might not be a bad idea. Personally, I'd probably go down the MPL route, although the GPL isn't particularly appropriate for Scarlet and can actually be ignored for the most part ...

Cheers,
Wol

Committing to Rust for kernel code

Posted Nov 22, 2023 19:03 UTC (Wed) by josh (subscriber, #17465) [Link] (26 responses)

Absolutely not. Copyleft is a good thing, let's not weaken it.

As a Rust developer, it makes me sad that rustc is permissively licensed. Just about the only *good* thing about it being permissively licensed is that that makes a good argument that there's no reason to need a second one. :)

Committing to Rust for kernel code

Posted Nov 22, 2023 19:15 UTC (Wed) by Wol (subscriber, #4433) [Link] (25 responses)

> Absolutely not. Copyleft is a good thing, let's not weaken it.

But *W*H*Y* is it a good thing? It makes large swathes of endeavour economically unviable.

Open Source that respects the four freedoms is fine too!

To my mind, Copyleft falls fairly seriously foul of the law of unintended consequences. Nature unfortunately is full of "pick two, pick any two" scenarios, and why is copyleft any different?

But anyways, I was merely responding to pizza's comment about moving away from copyleft stuff, and if it was a conspiracy theory, the easy way to carry out that conspiracy would be to make sure all kernel Rust code was liberally licenced.

Cheers,
Wol

Committing to Rust for kernel code

Posted Nov 22, 2023 21:03 UTC (Wed) by gdamjan (subscriber, #33634) [Link]

Isn't there FreeBSD for people wanting a BSD licensed kernel?

Leave our kernel GPL2

Committing to Rust for kernel code

Posted Nov 22, 2023 22:52 UTC (Wed) by Karellen (subscriber, #67644) [Link] (23 responses)

But *W*H*Y* is it a good thing?

Because ensuring that all users have the same freedoms is a good thing. Because allowing one set of users (3rd-party devs) to take the freedoms they have been given, away from other end-users, so that those end-users cannot make use of those freedoms themselves (or pay others to do so on their behalf), is - you know - unjust.

Because code is not scarce. It can be infinitely copied, and me sharing code I have received with you does not decrease the utility that that code has for me. Me giving you the same rights I was given, does not diminish mine.

The claim that "I am not free unless I have the freedom to take away other people's freedoms" is, IMHO, absurd.

It makes large swathes of endeavour economically unviable.

Maybe this being an issue is a problem with economics, not with code?

Committing to Rust for kernel code

Posted Nov 23, 2023 7:44 UTC (Thu) by Wol (subscriber, #4433) [Link] (19 responses)

> The claim that "I am not free unless I have the freedom to take away other people's freedoms" is, IMHO, absurd.

Yes it is absurd. Not least because it's circular, and applies *EQUALLY* to GPL vs MIT.

GPL takes away (in considerable part) developers' freedom to make a living. Even its pre-amble says it takes away freedoms, iirc.

If we, as developers, *choose* to pass on our MIT freedoms to our customers, then everyone gets the best of both worlds. The problem is, this is a social choice, a *freedom*, and, as I keep saying, this falls squarely into nature's "pick any two" conundrum. How do you square the circle? I don't think you can ...

Cheers,
Wol

Committing to Rust for kernel code

Posted Nov 23, 2023 8:21 UTC (Thu) by ssmith32 (subscriber, #72404) [Link] (11 responses)

What's "the freedom to make a living"?

Based on your past comments, I'm pretty sure my knee-jerk reading of that as an entitled "right to make a living" is most certainly off - that is, it's a genuine question..

Committing to Rust for kernel code

Posted Nov 23, 2023 13:14 UTC (Thu) by Wol (subscriber, #4433) [Link]

I think it's all down to the problem that freely available source code makes it hard to earn money from writing that source.

Sure, some companies (eg Red Hat) make money supporting that source.

While it's hard to put a finger on it, I just get the impression that more people make a living from OSS code that isn't Copyleft. And OSS makes it easier to sell the resulting software, even if you do "live by the spirit" and release the corresponding source.

Maybe I am a bit naive, but I believe most people agree with "live by the spirit", and if you do your best only to associate with others like that, OSS is much better than copyleft. OSS came before copyleft (in practice), and looks like outliving copyleft.

The difficulty is probably down to the fact that the software industry makes money from selling software (and the GPL interferes with this), while the world in general saves money by sharing software, and OSS helps this massively. Two completely opposing financial incentives, two completely mismatched mindsets.

Cheers,
Wol

Committing to Rust for kernel code

Posted Nov 23, 2023 13:18 UTC (Thu) by Wol (subscriber, #4433) [Link] (9 responses)

Just to add, I said the GPL takes away developer freedom. Because it quite openly seeks to ensure *end*user* freedom.

So software producers naturally favour OSS? Yep, people whos' business is software production prefer closed licences, but if software is a byproduct, OSS looks the best bet.

Cheers,
Wol

Committing to Rust for kernel code

Posted Nov 23, 2023 17:42 UTC (Thu) by ballombe (subscriber, #9523) [Link] (8 responses)

> Just to add, I said the GPL takes away developer freedom. Because it quite openly seeks to ensure *end*user* freedom.

Every developer is using far more code than what they contributed, so globally, the GPL is still a win for them.

Yes, you can say it is the "prisoner dilemma" all over and again and use MIT license to get an edge, but the "prisoner dilemma" terminology calls it "being selfish".

I will not go that far because release code under the MIT license is very generous but you see the point I making.

Committing to Rust for kernel code

Posted Nov 23, 2023 18:16 UTC (Thu) by Cyberax (✭ supporter ✭, #52523) [Link] (7 responses)

> Every developer is using far more code than what they contributed, so globally, the GPL is still a win for them.

Why? It's vice-versa. MIT is more advantageous for individual developers.

> Yes, you can say it is the "prisoner dilemma" all over and again and use MIT license to get an edge, but the "prisoner dilemma" terminology calls it "being selfish".

I think you might be confusing the "selfish" vs "selfless" choices here. GPL is the "selfish" choice, while MIT is a "selfless" choice that results in a better outcome if everyone cooperates.

Committing to Rust for kernel code

Posted Nov 23, 2023 19:41 UTC (Thu) by pbonzini (subscriber, #60935) [Link] (6 responses)

> MIT is more advantageous for individual developers.

Only if they are not interested in using any of Linux, GCC, glibc, GMP, GNOME, KDE, git, etc., free equivalents of which might or might not exist in a world without copyleft.

Committing to Rust for kernel code

Posted Nov 24, 2023 3:59 UTC (Fri) by Cyberax (✭ supporter ✭, #52523) [Link]

> free equivalents of which might or might not exist in a world without copyleft.

FreeBSD exists, LLVM exists, musl libc exists as well.

Also, it doesn't really change the fact that MIT is a more flexible license. By choosing it, developers prioritize the long-term well-being of the project, rather than a momentary gain (from people forced to open proprietary contributions).

Committing to Rust for kernel code

Posted Nov 24, 2023 9:17 UTC (Fri) by Wol (subscriber, #4433) [Link] (4 responses)

> glibc, GMP, GNOME, KDE, git

Linux didn't use glibc for a long time.

Dunno about GMP, but Gnome and KDE relied on X which was BSD. Didn't we have Lesstif? And git was written by Linus, who is famously a pragmatist and doesn't really agree with the GPL. He used it because it was available, and did what he wanted.

So I think pretty much EVERY one of your examples could easily exist in a world where the BSD lawsuit didn't happen, and Linux was never born.

What other differences such a world would involve I don't know.

Oh - and again as I keep repeating, in many cases there is NO REAL DIFFERENCE between GPL and MIT/BSD. What's the difference if you apply them to interpreted scripts? Zilch. What's the difference if you apply them to a p-code system like Pick/MV that has no concept of linking? Zilch.

That's why my licence of choice would probably be LGPL / MPL. The GPL brings nothing at all to the table, and the other two "actively encourage" share-and-share-alike. Plus, I work for an end-user (almost always have), so I don't feel the "make a living from software" pressure, despite being a 100% through and through developer.

Cheers,
Wol

Committing to Rust for kernel code

Posted Nov 27, 2023 18:00 UTC (Mon) by smoogen (subscriber, #97) [Link] (3 responses)

>> glibc, GMP, GNOME, KDE, git

> Linux didn't use glibc for a long time.

I am probably confused by what you meant here.. From a distribution point of view:

1. That 'long time' over the length of Linux operating systems is now very short. Most operating systems had switched to upstream glibc by 6-7 years out of the first distributions.. so out of 20 year lifetime, about 1/3 of the time?

2. The libraries shipped on many of the early distributions were either glibc derived OR were GPL2 based. I think there were a couple of BSD based ones but I don't remember who shipped them.

That said, the licensing of the new kernel code would be up to the writers of the code and would be limited to BSD or MIT to be GPLv2 compatible. [The Linux kernel has BSD and MIT code in it.]

The one thing I have seen with GPLv2 kernel code versus BSD code has been that short term effects BSD code can make money but it also tends to have a short 'shelf-life'. Updates to parts get stuck away inside various companies and you end up with multiple people solving the same problem over and over again. The kernel code tends to be seen and shared around a lot more.. and fixes come in from various places. BSD maintainers I have known usually grouse about how hard it is to get fixes for their stuff. They know in talking with programmer Y that XYZ company knows the bug, fixed the bug, and has code.. but because the code could also be used elsewhere, etc.. it would need so many legal checkmarks to get it out no one is going to do it. [The same does happen with GPLv2 code.. but it seems that there is more of a 'we can't use this elsewhere since its already GPLv2 so meh share it.']

However I know that is also just my experiences and world view clouding my view.

Committing to Rust for kernel code

Posted Nov 28, 2023 10:26 UTC (Tue) by paulj (subscriber, #341) [Link] (2 responses)

Speaking of pre-libc.... there seems to be nearly 0 information available on the Internet - at least, easily found via the indexes of my favourite search engines of the early (pre GNU libc) Linux libc.

What was its origin and history again?

Committing to Rust for kernel code

Posted Nov 28, 2023 11:06 UTC (Tue) by paulj (subscriber, #341) [Link] (1 responses)

Self answer, best source seems to be the Linux "libc" man page: https://man7.org/linux/man-pages/man7/libc.7.html - linux libc was a GNU libc 1.x fork.

Committing to Rust for kernel code

Posted Nov 28, 2023 11:51 UTC (Tue) by Wol (subscriber, #4433) [Link]

You could always search for libc5 - the last linux libc. glibc2 became "libc6".

I've got some old software I would like to try and get running again (Y2K era) that needs libc5.

Cheers,
Wol

Committing to Rust for kernel code

Posted Nov 23, 2023 8:27 UTC (Thu) by LtWorf (subscriber, #124958) [Link] (6 responses)

> GPL takes away (in considerable part) developers' freedom to make a living.

Au contraire :)

MIT takes away that possibility by letting for example google or apple use that code. But gpl3 or agpl would force most companies to reimplement, and consequently hire someone to re-do the work.

The way I see it, with GPL the community is getting an advantage, with MIT some stakeholders are getting an advantage.

Developers who use GPL can also double license if they want.

What you mean is that GPL is taking away the freedom to make a living off of someone else's work without contributing anything.

Arguably also getting on a drakkar and raiding england is a way to make a living that current pesky laws have forbidden. Perhaps not all ways of making a living are equally good?

Committing to Rust for kernel code

Posted Nov 24, 2023 19:55 UTC (Fri) by Wol (subscriber, #4433) [Link] (5 responses)

> Arguably also getting on a drakkar and raiding england is a way to make a living that current pesky laws have forbidden. Perhaps not all ways of making a living are equally good?

The difference is do you want to live in a commune, or a communist state. Both are supposedly "share and share alike". But a commune is from choice, communism is imposed.

From the *givers* point of view, MIT and GPL are pretty much the same thing - you chose to give it away. From the recipient's pov, MIT gives you that same choice, that same freedom. GPL forces you to give your stuff away, under threat of lawsuit ...

(Yes, I know, only if you choose to share what others gave you, but the GPL applies coercion, MIT gives you choice)

Cheers,
Wol

Committing to Rust for kernel code

Posted Nov 25, 2023 7:32 UTC (Sat) by LtWorf (subscriber, #124958) [Link] (4 responses)

GPL is not imposed. If you use it it was because you have decided to somehow obtain the software, and you have no obligation of doing so.

You can write your own, pay someone else to do it, just not use that particular software. If you are ok with windows having an EULA, I don't see what your problem is with a much less restrictive GPL license.

> From the *givers* point of view, MIT and GPL are pretty much the same thing

From wrong axioms, a wrong proof follows. Unsurprisingly.

From the author's point of view, GPL means a much greater chance of getting improvements and contributions. See webkit, it exists because apple had to release it because KHTML was LGPL. Otherwise apple would have just never released their modifications. I do this assumption because the rest of safari was never open sourced.

So the givers of LGPL KHTML got webkit (wich wasn't that nice, because it was not a direct replacement), but it's more than with MIT.

If you are scared of the legal threat of the GPL and feel constrained by the clause that you must pass on the same freedoms, your intentions were never good to begin with; and as an author of free software I'm completely ok with ill intentioned people never using my software and interacting with me via bugreports.

(Of course in reality the ill intentioned people open bugreports to inform me of how dumb/evil I am for not letting them use my work to take freedom away from others… So I wrote a code of conduct that forbids this behaviour and I can at least instantly ban them).

> (Yes, I know, only if you choose to share what others gave you, but the GPL applies coercion, MIT gives you choice)

And seeing how the former CEO of github was pushing for MIT license (https://www.youtube.com/watch?v=-bAAlPXB2-c), microsoft continuing on the same path, apple avoiding GPL3, google completely forbidding AGPL… it seems that humanity isn't enlightened enough yet to not need the coercion.

Committing to Rust for kernel code

Posted Nov 25, 2023 11:39 UTC (Sat) by Wol (subscriber, #4433) [Link] (3 responses)

> > From the *givers* point of view, MIT and GPL are pretty much the same thing

> From wrong axioms, a wrong proof follows. Unsurprisingly.

Where's the difference? If it's GPL, commercially minded people won't use it, and you don't get anything back.

If it's MIT, commercially minded people don't contribute back. (According to you, at least ...)

The mindset that says "pay forward, pay back", will contribute regardless of licence.

As I say, what's the difference?

And if the GPL has no teeth (because, like in my case, it's simply an inappropriate licence), again what's the difference?

Cheers,
Wol

Committing to Rust for kernel code

Posted Nov 25, 2023 13:19 UTC (Sat) by LtWorf (subscriber, #124958) [Link] (2 responses)

> Where's the difference? If it's GPL, commercially minded people won't use it, and you don't get anything back.

Wrong premise again? Since Safari and Android do in fact exist, it's rather easy to show that you are incorrect.

I will stop this here because there can be no agreement if we can't even acknowledge real facts.

Committing to Rust for kernel code

Posted Nov 25, 2023 17:36 UTC (Sat) by Wol (subscriber, #4433) [Link] (1 responses)

> > Where's the difference? If it's GPL, commercially minded people won't use it, and you don't get anything back.

> Wrong premise again? Since Safari and Android do in fact exist, it's rather easy to show that you are incorrect.

Yep, they exist. However, I can't speak for Safari/Apple because I have nothing to do with that ecosystem, but don't Apple refuse to have anything whatsoever to do with GPL3? And as for Android, doesn't Google have a ban on GPL user-space? Safari and Android look extremely like exceptions to me ...

> I will stop this here because there can be no agreement if we can't even acknowledge real facts.

To misquote PJ, when dealing with real life, facts are squishy. Heck, even with your two examples, the companies involved are openly hostile to the GPL - they don't have anything to do with it if they can help it .. (Unless, in Google's case, I suspect they're quite happy to dump abandoned projects as GPL ...)

Cheers,
Wol

Committing to Rust for kernel code

Posted Nov 29, 2023 12:30 UTC (Wed) by nim-nim (subscriber, #34454) [Link]

> don't Apple refuse to have anything whatsoever to do with GPL3

Apple is not the whole world

In fact a *huge* number of companies (both on the producer and user side) have been using (AL)GPL for years and are starting to realise the “communist hell” FUD is just FUD. And they’re complaining Red Hat is not free enough.

The loudest free software promoters for years have not been @FSF but VC players that try to use dual licensing tricks and other open-source-enabled antifeatures to force their customers to do things they have no interest in.

Committing to Rust for kernel code

Posted Nov 24, 2023 14:27 UTC (Fri) by orangetuba (guest, #168176) [Link] (2 responses)

Do you also think that restaurants should publish all their recipes, and that not doing so is equivalent to limiting other people's freedom?

Committing to Rust for kernel code

Posted Dec 1, 2023 23:17 UTC (Fri) by jschrod (subscriber, #1646) [Link] (1 responses)

Well, I'm a foodie. In my experience all chefs from Michelin stared restaurants are readily willing to share their receipts.

The problem at hand: there are very few people who can reproduce them at home. And those who can do so, usually create their own receipts - by building upon the ideas of those others.

Actually, hanging out with chefs is astonishing similar to hanging around with free software enthusiasts.

Committing to Rust for kernel code

Posted Dec 2, 2023 9:36 UTC (Sat) by atnot (guest, #124910) [Link]

Not to get too far from the point, but there's another aspect to it.

It's perfectly possible for most decent home cooks to make a michelin star grade meal at home with a bit of practice... once. That's a large part of the joy of being a home cook, really.

But the actual skill and difficulty in a restaurant is in being able to deliver that type of quality consistently at scale hundreds of times a day, every week, for years. Every week will have different weather, shifting suppliers, fresh ingredients: a new recipie. Adjusting to all that requires immense ability and intuition from a whole team.

Committing to Rust for kernel code

Posted Nov 22, 2023 19:01 UTC (Wed) by Phantom_Hoover (subscriber, #167627) [Link] (1 responses)

> A primary goal of this Rust-in-Linux effort is to change that via a kernel-of-thesius approach.

Is this something supported by statements people have actually made, or something you’ve invented to support your grand conspiracy theory of a purge of copyleft software? Why would this conspiracy need to undertake the expense of rewriting the kernel in Rust to break its dependency on GCC when the same thing could be and has been achieved much more easily just by building it with Clang?

Committing to Rust for kernel code

Posted Nov 22, 2023 19:26 UTC (Wed) by pizza (subscriber, #46) [Link]

>> A primary goal of this Rust-in-Linux effort is to change that via a kernel-of-thesius approach.
> Is this something supported by statements people have actually made, or something you’ve invented

As I said in the message you responded to, it's the only logical goal for all of this effort, especially in light of the article's numerous cited examples of the fundamental conflicts between the C and Rust camps in the Kernel.

(This is also an opinion shared by numerous other kernel developers, fueling much of the skepticism in the article's citations!)

Mind you, I'm not saying this goal is inherently a bad thing, but it doesn't make a whole lot of sense to go through all of this effort and the immense ongoing maintenance burden of having two separate camps, bifrucating hardware support based on tooling and subtly different APIs unless the goal is to eliminate C from the highly complex "core" subsystems like MM, VFS, and so forth.

Committing to Rust for kernel code

Posted Nov 22, 2023 19:28 UTC (Wed) by pbonzini (subscriber, #60935) [Link] (18 responses)

Conversion of Linux to 100% Rust is not going to happen. It is simply impossible, also given the hard to test nature of hardware-dependent kernel code.

On the other hand, requiring it for most distro builds is definitely going to happen. My bet is 2 years before distro builds use it and 3-5 years before all of us run Rust code. But it may be earlier too.

Committing to Rust for kernel code

Posted Nov 22, 2023 19:39 UTC (Wed) by pizza (subscriber, #46) [Link] (17 responses)

> Conversion of Linux to 100% Rust is not going to happen. It is simply impossible, also given the hard to test nature of hardware-dependent kernel code.

FWIW I generally agree, but that doesn't mean that C will remain the "primary/core" language. Instead it will eventually be relegated to the long tail of "legacy" device drivers.

> On the other hand, requiring it for most distro builds is definitely going to happen. My bet is 2 years before distro builds use it and 3-5 years before all of us run Rust code. But it may be earlier too.

I'll be surprised if it takes even a year, probably starting with Fedora wanting to run out of the box on modern Apple hardware. (That's going to be a fun change proposal discussion!) Meanwhile the next LTS/EL cycle is all but guaranteed to require Rust.

As I said, "Rust is optional" makes no logical sense, giving you the worst of both worlds. Either commit to it as a core feature, or drop it entirely.

Committing to Rust for kernel code

Posted Nov 22, 2023 19:48 UTC (Wed) by Conan_Kudo (subscriber, #103240) [Link] (13 responses)

Fedora Asahi kernel maintainer here. I will be the first to say that given how Rust-for-Linux is governed and developing, I don't want to turn Rust on in the Fedora kernel yet. Having the kernel build break every six weeks is basically madness, and the lack of priority to eliminate usage of nightly/unstable Rust features and the difficulty getting them to care about tracking newer Rust compiler versions has soured me significantly on the effort.

With the lack of a proper specification; lack of stabilized interfaces; lack of stability for even compiling the code; and the lack of interest from kernel developers to review the Rust abstraction patches needed to bring in Rust-based drivers, I do not think enabling Rust in a distribution kernel would be reasonable to consider for a few years.

Committing to Rust for kernel code

Posted Nov 22, 2023 20:13 UTC (Wed) by pizza (subscriber, #46) [Link]

> Having the kernel build break every six weeks is basically madness, and the lack of priority to eliminate usage of nightly/unstable Rust features and the difficulty getting them to care about tracking newer Rust compiler versions has soured me significantly on the effort. [...] the lack of a proper specification; lack of stabilized interfaces; lack of stability for even compiling the code; and the lack of interest from kernel developers to review the Rust abstraction patches [...]

This reminds me of the ultimately-unsuccessful Van Jacobson Network Channels effort, only with many more moving parts.

....Much like the VJNC, RoL has to commit to building out enough of the new approach to determine if it will be viable in the long run. But unlike VJNC, this adds a major cultural clash into the mix, and that might be what ultimately dooms this whole experiment..

(Thanks for your reply, btw)

Committing to Rust for kernel code

Posted Nov 23, 2023 0:00 UTC (Thu) by ojeda (subscriber, #143370) [Link] (11 responses)

> Having the kernel build break every six weeks is basically madness

What do you mean by "break"?

> the lack of priority to eliminate usage of nightly/unstable Rust features

This is false. It is, in fact, a priority for us: https://rust-for-linux.com/unstable-features.

> the difficulty getting them to care about tracking newer Rust compiler versions has soured me significantly on the effort.

This is also false. We are, in fact, tracking the latest version, as anybody can see in the kernel Git log. See also https://rust-for-linux.com/rust-version-policy.

> lack of a proper specification

I am not sure what you mean by this (and other points you make in this second paragraph), but if you are talking about a Rust language specification, then that is not really on us. In any case, the Rust project has decided to start working on an official specification for the Rust language. Ferrocene has one too.

> the lack of interest from kernel developers to review the Rust abstraction patches needed to bring in Rust-based drivers

Different subsystems have different timelines, requirements, bandwidth and so on. You cannot expect every kernel maintainer to be on board or have time to work on Rust this early on. Having said that, as I mentioned in the summit, during this last year we have seen an increased interest from kernel maintainers, and in fact some of them have already been very helpful and supportive.

> I do not think enabling Rust in a distribution kernel would be reasonable to consider for a few years.

A few major distributions told me it should be possible for them to support Rust for Linux in the future. Ubuntu, for instance, provides packages with the Rust toolchain needed for the kernel.

Committing to Rust for kernel code

Posted Nov 23, 2023 0:25 UTC (Thu) by Conan_Kudo (subscriber, #103240) [Link] (10 responses)

>> Having the kernel build break every six weeks is basically madness
>
> What do you mean by "break"?

It fails to compile because something has broken in rustc in some way, or the unstable feature has changed and broken something, or both. It happens without fail.

>> the lack of priority to eliminate usage of nightly/unstable Rust features
>
> This is false. It is, in fact, a priority for us: https://rust-for-linux.com/unstable-features.

You still allow *new* unstable features to be used. That list will keep growing. And in fact, I know it will once the DRM stuff is incorporated into mainline RFL (whenever that happens). It's only a priority once you start saying "no" to adding more of them. That can't happen for a while yet since the various subsystems don't yet have Rust abstractions in mainline.

>> lack of a proper specification
>
> I am not sure what you mean by this (and other points you make in this second paragraph), but if you are talking about a Rust language specification, then that is not really on us. In any case, the Rust project has decided to start working on an official specification for the Rust language. Ferrocene has one too.

I mean that you do not have a specification of how you want to handle Rust *in* Linux itself. This makes it really hard for people to figure out how to grade it and grok it when patches are being submitted to build new abstractions or kernel modules.

>> the lack of interest from kernel developers to review the Rust abstraction patches needed to bring in Rust-based drivers
>
> Different subsystems have different timelines, requirements, bandwidth and so on. You cannot expect every kernel maintainer to be on board or have time to work on Rust this early on. Having said that, as I mentioned in the summit, during this last year we have seen an increased interest from kernel maintainers, and in fact some of them have already been very helpful and supportive.

No, that's true. But I *can* expect the ones that *are* already doing Rust in other places to do more than nothing in the kernel. *My* patch load keeps growing because *nothing* has made it upstream in a year. It's extremely frustrating.

>> I do not think enabling Rust in a distribution kernel would be reasonable to consider for a few years.
>
> A few major distributions told me it should be possible for them to support Rust for Linux in the future. Ubuntu, for instance, provides packages with the Rust toolchain needed for the kernel.

I am shocked. I was also shocked when you said you reached out to all the major distributions, because I have not seen you reach out to Fedora *at all*.

Insofar as Ubuntu shipping it, it's pretty easy when there's nothing to ship.

The Fedora Asahi Remix kernel has one of the most significant Rust-based kernel modules (the Asahi DRM driver), and it can't even build on Rust 1.73 because something is broken in rustc itself (though nobody cares because it's broken somewhere related to a nightly/unstable feature). My next rebase will be the next time I try with Rust 1.74, and I'm dreading it.

I have recommended to upstream Fedora to *not* enable it because it's so risky to enable right now. Since we track upstream Rust on their compilers, Linux Rust code is busted all the time.

Committing to Rust for kernel code

Posted Nov 24, 2023 2:33 UTC (Fri) by ojeda (subscriber, #143370) [Link] (8 responses)

> It fails to compile because something has broken in rustc in some way, or the unstable feature has changed and broken something, or both. It happens without fail.

The kernel only supports a single version of rustc for the time being (as the page mentioned above explains). Obviously, if you are using an unexpected compiler and also have out-of-tree code with extra unstable features, then you will need to handle the upgrade work yourself (which is why upstreaming would help).

One of the reasons why we started tracking the latest Rust release more closely is because it could help some very early users, including you. So I am not sure what you still expect us to do — I already told you back then that the kernel tracking the very latest Rust release would not solve all your problems (and, in fact, if you are dealing with a ton of out-of-tree code, then it may have been best for you that the kernel did not change versions that often and simply provide the right toolchain on your side).

> You still allow *new* unstable features to be used. That list will keep growing. And in fact, I know it will once the DRM stuff is incorporated into mainline RFL (whenever that happens). It's only a priority once you start saying "no" to adding more of them. That can't happen for a while yet since the various subsystems don't yet have Rust abstractions in mainline.

Please read the second page I mentioned earlier. You seem to believe that the features included in the initial minimal Rust merge were the only ones that we would need to use, forever. This is not the case, and we even have an issue (started in 2020) in our tracker listing the features we expect to need (linked from the other page too).

Moreover, it is also not the case that all features are treated equally (e.g. not all have the same timelines or the same degree of stability) nor that we add them just because we fancy them (the default answer is "no", in fact), which is what one could understand from your different comments in this article. Again, please refer to the page above.

Furthermore, whether some code is in mainline or not is irrelevant. You seem to believe that any maintainer in the kernel is able to start using any feature they like, which is not true. On top of that, for code outside abstractions, we are way stricter on the use of unstable features.

Finally, we are doing our best to work with the Rust project (and other entities) to try to get to the point where we can eventually declare a minimum version. So I really do not appreciate that you imply we are somehow careless about our usage of unstable features.

> I mean that you do not have a specification of how you want to handle Rust *in* Linux itself. This makes it really hard for people to figure out how to grade it and grok it when patches are being submitted to build new abstractions or kernel modules.

We are happy to help here, but it is unclear what you mean by "specification". What would you expect to see there? If you mean a contribution guide, we already have one among other documentation. Of course, we are still improving it and completing it, but others have already found it useful and successfully submitted patches.

> But I *can* expect the ones that *are* already doing Rust in other places to do more than nothing in the kernel. *My* patch load keeps growing because *nothing* has made it upstream in a year. It's extremely frustrating.

Why would you expect that? I am confused. Someone may like Rust, and even use it elsewhere (outside the kernel or in other areas of the kernel), but that does not mean they agree with using it within the kernel or in a particular area of it, or that they have the bandwidth to help you, or the expertise in a particular topic, or they may simply only work in the kernel as part of their job which may not include time for that, etc.

Anyway, I understand it can be frustrating, but it is what it is. Kernel maintainers, even those willing to help you, cannot simply merge code for the only reason of reducing your patch load. And, in case that is part of the confusion, kernel maintainers cannot randomly merge changes for areas they do not maintain. (To be clear, I don't know who you are referring to, i.e. I am not talking about anybody in particular.)

> I am shocked. I was also shocked when you said you reached out to all the major distributions, because I have not seen you reach out to Fedora *at all*.

Fedora already tracks the latest Rust compiler and Josh Stone has been involved in our issue tracker discussing the versioning policy (just like you were — see the first point).

And to be clear, I reached out to some that seemed to be able to provide a recent enough Rust release in their rolling user-facing releases. Something like Debian stable will obviously take longer, which is why when it came up in the summit I suggested providing the toolchain at kernel.org as another alternative that could perhaps help some people. Distributions could also provide a special package, like Ubuntu is doing — that would be great too and we would appreciate it.

> Insofar as Ubuntu shipping it, it's pretty easy when there's nothing to ship.

The point was that Ubuntu provides the toolchain required by the kernel, thus compiling the kernel should always work for them regardless of how much code or unstable features are in use in mainline.

> The Fedora Asahi Remix kernel has one of the most significant Rust-based kernel modules (the Asahi DRM driver), and it can't even build on Rust 1.73 because something is broken in rustc itself (though nobody cares because it's broken somewhere related to a nightly/unstable feature). My next rebase will be the next time I try with Rust 1.74, and I'm dreading it.

We cannot promise out-of-tree code compiles. The best route is to get the code upstreamed and use the compiler version expected by the kernel. The DRM maintainers are supportive of Rust, so that is definitely an advantage in your case.

I don't know why you say "nobody cares" or why you claim "something is broken in rustc itself", though. Please see the points above.

> I have recommended to upstream Fedora to *not* enable it because it's so risky to enable right now.

If by "risky" you mean "I get compile errors because I am carrying out-of-tree code with extra unstable features and use an unexpected compiler on top", sure. In any case, I would also not enable it until there is an actual in-tree module that their users actually need and the support is deemed production-ready. So for the time being I agree with your recommendation, even if for different reasons.

> Since we track upstream Rust on their compilers, Linux Rust code is busted all the time.

Linux Rust code is definitely not "busted". Again, if you are using out-of-tree code and an unexpected compiler on top of that, then that is something you will need to handle yourself.

Sincerely, it sounds to me like you are doing some things on your own, facing some difficulties for some time and then deciding that the best approach is to complain here. We already tried to help you in the past, and it seems you did not notice we started tracking the latest release. I think a better approach would be reaching out to us to see if we can help you further or at least to understand each other's context/constraints better.

Committing to Rust for kernel code

Posted Nov 24, 2023 12:52 UTC (Fri) by Conan_Kudo (subscriber, #103240) [Link] (5 responses)

Most of the problems are in the forked "alloc" crate that's in-tree, not in the Asahi DRM driver code. The website also says not to send patches for making the tree compatible with newer Rust compilers, so we don't. Instead, we have patches downstream in the Asahi tree to make adjustments to fix it for newer Rust versions.

My complaints around my patch load are centered around two things: you don't want fixes for newer Rust versions *and* DRM (despite being mostly enthusiastic about Rust) has been stalling on the reviews. The latter issue is apparently resolving itself now, so we may see progress on that front. But I'm still not happy about the fact you do not accept patches for fixing the code to work on newer Rust versions.

Committing to Rust for kernel code

Posted Nov 25, 2023 16:58 UTC (Sat) by ojeda (subscriber, #143370) [Link] (4 responses)

> Most of the problems are in the forked "alloc" crate that's in-tree, not in the Asahi DRM driver code.

Glad to hear that; then if you take the upgrades we do, most of your problems should go away.

> The website also says not to send patches for making the tree compatible with newer Rust compilers, so we don't. Instead, we have patches downstream in the Asahi tree to make adjustments to fix it for newer Rust versions. (...) you don't want fixes for newer Rust versions

The website asks not to send upgrades to newer versions because we are handling those on our side already. What we recommend is that users take the upgrades that we put upstream, and then you do not need to deal with alloc anymore.

> you don't want fixes for newer Rust versions (...) But I'm still not happy about the fact you do not accept patches for fixing the code to work on newer Rust versions.

Again, we support a single compiler version for the moment, and we are already upgrading to newer versions if there are no blockers. Thus there are no "fixes" needed (modulo mistakes in the upgrade, of course).

I suspect you may be talking about supporting multiple versions and wanting to add workarounds for that, but that is a different discussion/approach altogether, and I don't recall you suggesting that? In fact, in that GitHub issue, you said:

"I think that as long as you use unstable features, you should be tracking the latest Rust compiler at their lifecycle. Once you're not using unstable features anymore, then widening the range of compilers supported is a reasonable conversation to have."

Committing to Rust for kernel code

Posted Nov 25, 2023 17:21 UTC (Sat) by Conan_Kudo (subscriber, #103240) [Link] (3 responses)

The approach we're currently taking is to do the minimal effort to enable the Rust compiler version shipped in Fedora, which is nearly always ahead of what Rust-for-Linux is tracking. Keep in mind that we're shipping new versions of rustc shortly after upstream releases them. I don't particularly care about supporting multiple versions of Rust, but depending on what the change is, it may look like it might work for multiple Rust versions even if that's completely untested.

Committing to Rust for kernel code

Posted Nov 27, 2023 15:48 UTC (Mon) by ojeda (subscriber, #143370) [Link] (2 responses)

I am aware of all that, which is precisely why I wrote what I wrote.

I am still not understanding what you are trying to suggest. That we take "completely untested" patches that attempt to support multiple versions? That is not how things work.

And even if we decided to support multiple versions, that would still not solve your problems, since you would still be ahead.

Committing to Rust for kernel code

Posted Nov 27, 2023 16:10 UTC (Mon) by Conan_Kudo (subscriber, #103240) [Link] (1 responses)

We are doing that approach downstream because we need to. Obviously for upstream purposes, we'd do things differently. Our downstream patches are intended to enable us to build with the latest Rust compiler available in Fedora. Your approach with RfL is mostly fine, it's just we're not at a place where I would be comfortable recommending it to be enabled in a mainline distro kernel. The forked alloc crate issue has to be resolved first.

Committing to Rust for kernel code

Posted Nov 27, 2023 16:26 UTC (Mon) by ojeda (subscriber, #143370) [Link]

> Your approach with RfL is mostly fine,

As I told you both here and in Zulip, please explain what is your proposal for improvement. If you do not have one, please do not assume or imply that things could be done better.

The Rust versions we pick may not be the best for you (in fact, they are already the best ones we can pick for you), but that does not mean Rust for Linux is governed or developed in a bad way given the current constraints.

> it's just we're not at a place where I would be comfortable recommending it to be enabled in a mainline distro kernel.

Like I told you in Zulip, I am not sure why this is relevant. Whatever you recommend does not change our constraints.

I also already told you that we ourselves tell general purpose distributions that it is early to enable the upstream Rust support.

> The forked alloc crate issue has to be resolved first.

No, the reason is not alloc. The fact that you say so concerns me, since that is an implementation detail for users.

The actual reasons are, for instance, that there are no users in-tree, MODVERSIONS, etc.

Committing to Rust for kernel code

Posted Nov 25, 2023 8:02 UTC (Sat) by marcan (guest, #103032) [Link] (1 responses)

Hey Miguel,

I just opened a thread on the Zulip to talk about this. Conan doesn't have too much visibility into the actual work going into our downstream branch Rust support, but I do. And indeed, the biggest problem is 100% upstream: it's the `alloc` crate. That's what is breaking with almost every compiler version bump, and I haven't seen any discussion or plans for how that will be resolved yet.

Upstreaming, the small handful of features used in the other crates, etc. are all things with solid plans going forward, and I look forward to the future for Rust for Linux. But as long as `alloc` is using 100+ unstable features directly upstream, all the work to eventually drop the dozen features used in the `kernel` crate is rather useless. Being able to unpin the Rust compiler version is very important for distros to start taking this seriously and reduce friction enabling Rust support, so I hope there is some plan to eventually solve this. Maybe there is and I just haven't found it? Would be happy to hear your thoughts :)

Committing to Rust for kernel code

Posted Nov 25, 2023 15:44 UTC (Sat) by ojeda (subscriber, #143370) [Link]

The alloc topic has been in discussion since essentially the beginning of the project. The original plan discussed with upstream Rust (and others) is documented in-tree, please see the rust/alloc/README.md file. In short, the alloc fork is meant to be temporary and the discussions with upstream Rust are ongoing.

Committing to Rust for kernel code

Posted Nov 24, 2023 11:38 UTC (Fri) by khim (subscriber, #9252) [Link]

> though nobody cares because it's broken somewhere related to a nightly/unstable feature

Also, please, note that this is gross exaggeration: while unstable features can be deliberately changed at any time (that's why they are marked “unstable”, after all) if something is broken by accident then rustc developers take bug-reports seriously.

Have you tried to create in issue against rustc? Yes, they may ask you to change your code, if you use unstable feature, but “nobody cares” is the exact opposite from what I observed.

Firefox and many other large companies use Rust unstable features — do you think it would have worked for them if nightly features were in the state of being perpetually broken?

And if you have filed the issue then looking on it would be better than talking about abstract issues in vacuum.

Committing to Rust for kernel code

Posted Nov 23, 2023 14:01 UTC (Thu) by wtarreau (subscriber, #51152) [Link] (2 responses)

> I'll be surprised if it takes even a year, probably starting with Fedora wanting to run out of the box on modern Apple hardware. (That's going to be a fun change proposal discussion!) Meanwhile the next LTS/EL cycle is all but guaranteed to require Rust.
> As I said, "Rust is optional" makes no logical sense, giving you the worst of both worlds. Either commit to it as a core feature, or drop it entirely.

In a world where specs and compilers are distinct and evolve at their own rhythm, it would be true. But with a fast moving target which evolves to address the limitations making its use hard to impossible in the kernel for various purposes, and forcing periodic major upgrades, it will quickly become a huge mess. Gcc that didn't evolve that fast has already been quite a mess. And it did change only every few years. I was part of those who had to work on making late kernel 2.4 build with gcc-3.4 because that's what users had on their distros... In the end we only addressed two archs and the most visible stuff IIRC.

I can't imagine what it will look like in just one year with the current pace. Users will be forced to perform major updates from 6.6 to 7.1 just because "ah that old driver you're using, no, we're not maintaining it anymore, it required and old version of Rust that was not convenient for us, why don't you upgrade your LTS instead?". So that would basically mean no more long term maintenance and even more breakage in field. Sure, for plenty of use cases, nobody cares, machines are short-lived, deployed by automated scripts, and totally untuned. But for other use cases where it takes time to validate a version and adapt to it, it's another story.

Another thing is to observe how long rarely used drivers will be maintained. With a language that constantly evolves you can have change of behaviors or guarantees over time, possibly with nobody touching the driver for a while to adapt to the new version as long as it builds fine. Testing driver for hardware is very hard so that doesn't happen often. And possibly that some drivers will not build anymore after a few toolchain upgrades, and lacking a maintainer they will simply be dropped from the tree.

Time will tell, anyway, that train has already started.

Committing to Rust for kernel code

Posted Nov 23, 2023 18:29 UTC (Thu) by mathstuf (subscriber, #69389) [Link] (1 responses)

> Users will be forced to perform major updates from 6.6 to 7.1 just because "ah that old driver you're using, no, we're not maintaining it anymore, it required and old version of Rust that was not convenient for us, why don't you upgrade your LTS instead?".

Who is offering LTS kernels but not supporting the drivers they ship with it? IIUC, the kernel community isn't going to say "hey, you want to maintain this driver but we don't want you to, so we're ripping it out" is it?

> But for other use cases where it takes time to validate a version and adapt to it, it's another story.

Again, why would maintenance on the stable branch of the driver not be a suitable solution here? Presumably with the attitude the developers have in your scenario, they're not backporting anything anyways and not worried about conflicts.

Committing to Rust for kernel code

Posted Nov 24, 2023 8:42 UTC (Fri) by wtarreau (subscriber, #51152) [Link]

I'm speaking about implications of a fast-moving toolchain. When a toolchain moves fast the code can diverge quickly, to reach a point where backports are extremely complicated because sometimes they rely on a lot of new infrastructure. We've seen issues in the past where backporting fixes was a real mess. I'm thinking about the IRET bug for example, which required significant adaptations to the old code, backporting some infrastructure stuff while trying hard not to break anything. We could estimate ourselves lucky that the asm language did not evolve over all these years and that the infrastructure parts could be backported without other particular dependencies. But when you don't have all the underlying infrastructure available in older releases because it came with newer versions of the language, it's not the same game anymore. I was definitely not speaking about developers being lazy, just difficulties caused by dependencies on an updated version of a toolchain. That's not something we're used to with gcc, in the worst case it just requires less clean code.

Committing to Rust for kernel code

Posted Nov 23, 2023 13:43 UTC (Thu) by wtarreau (subscriber, #51152) [Link] (6 responses)

Well, the thing is that it will take more than 20 years and every 10 years new languages appear with their new fans who want to use them for everything and fix all of the world's remaining problems. Thus the migration from Rust to FooBar will start before C to Rust has completed and Rust has finally reached a stable state or gets two compilers to agree on what is safe and what is not. And no, they won't do C to FooBar because by then the remaining C will still be C precisely because it's too difficult to turn into languages where the compiler knows better than the developer. So in 10 years we'll have C, Rust and FooBar with even more maintenance fragmentation.

Committing to Rust for kernel code

Posted Nov 23, 2023 14:09 UTC (Thu) by farnz (subscriber, #17727) [Link] (2 responses)

The Linux kernel started over 30 years ago in C and assembly. If every 10 years, the kernel starts moving to a new language, but doesn't complete the migration before moving onto the next shiny, what are the 2 to 3 languages that it's using beyond C and assembly, and isn't yet finished migrating to?

Committing to Rust for kernel code

Posted Nov 23, 2023 16:18 UTC (Thu) by wtarreau (subscriber, #51152) [Link] (1 responses)

Till now nobody claimed to replace the one in place.

Committing to Rust for kernel code

Posted Nov 23, 2023 16:25 UTC (Thu) by farnz (subscriber, #17727) [Link]

There are several languages I've worked with that have claimed to be C replacements in the 30 years of the kernel using C. None of them got adopted, even with a group of people pushing for people to replace C with their language of choice. What makes you think that, assuming the kernel does adopt Rust, the next language to come along will be adopted, when C++, D, and others were not?

Committing to Rust for kernel code

Posted Nov 23, 2023 15:41 UTC (Thu) by beagnach (guest, #32987) [Link] (2 responses)

> every 10 years new languages appear with their new fans who want to use them for everything and fix all of the world's remaining problems.

In the 30+ years of Linux kernel development this is the first time a new language has been introduced, so I don't think your point stands.

Committing to Rust for kernel code

Posted Nov 23, 2023 20:26 UTC (Thu) by sfeam (subscriber, #2841) [Link] (1 responses)

Well, there was BPF...

Committing to Rust for kernel code

Posted Nov 24, 2023 15:27 UTC (Fri) by beagnach (guest, #32987) [Link]

Yes, but we all know that's not relevant to the discussion at hand because no one could seriously advocate for it to replace C for general kernel development.

Committing to Rust for kernel code

Posted Nov 27, 2023 1:14 UTC (Mon) by motk (subscriber, #51120) [Link]

Could we not have this conversation every other week just for once?

Committing to Rust for kernel code

Posted Nov 23, 2023 14:41 UTC (Thu) by linusw (subscriber, #40300) [Link] (4 responses)

I would rather say the approach is BOTH/AND not EITHER/OR.

Everybody will be required to be bilingual and it is not just going one way; e.g in the event that a reviewer request a feature in a newly submitted "leaf driver" (for something like phylib) to rather be moved to the core, then that means that the developer submitting that Rust code has to go and reimplement that feature in the core using C.

People grossly overestimate how hard Rust is, it's a trivial language to learn. All it does is deliver what Algol 68 and PASCAL delivered, but C omitted when it comes to spatial memory safety, then some temporal safety is added by nudging immutable variables and pure functions where possible inspired by misc functional languages. The rest is syntax. Gordon was right saying the language contains "nothing new" really.

In the device tree bindings that almost all driver maintaines have to deal with, we have to use YAML which is a *WAY* harder language to learn than Rust. Yet we just learn it, so I don't see the problem here.

Committing to Rust for kernel code

Posted Nov 23, 2023 19:39 UTC (Thu) by cozzyd (guest, #110972) [Link]

I don't think many people will agree that YAML is a more difficult language to learn than rust.

Committing to Rust for kernel code

Posted Nov 23, 2023 19:48 UTC (Thu) by pbonzini (subscriber, #60935) [Link] (2 responses)

Rust is trivial to learn and insanely hard to master, let's not kid ourselves. Most maintainers in the kernel could not write (and could barely read) the code for some of the bindings. See for example the workqueue bindings, https://lwn.net/Articles/932271/ (might not be the last version, the point still stands).

The way it works is to trust that the people who contributed them, or someone else, will be around to help. Fortunately the Linux APIs are relatively stable, and changes are less frequent than people think, but I think it's reasonable at least for now to have even experienced C programmers ask for help, because introducing technical debt would be worse.

Committing to Rust for kernel code

Posted Nov 24, 2023 20:02 UTC (Fri) by tlamp (subscriber, #108540) [Link] (1 responses)

IME the hardest part is writing good and rust-idiomatic bindings, but once that hard work is done (yes, often by experts) working with them can be done by non-experts too.

ps. How's QEMU's stance on rust? Our backup driver [0][1] would surely look a bit nicer with more native support.

[0]: https://git.proxmox.com/?p=proxmox-backup-qemu.git;a=summary
[1]: https://git.proxmox.com/?p=pve-qemu.git;a=blob;f=debian/p...

Committing to Rust for kernel code

Posted Nov 25, 2023 14:04 UTC (Sat) by pm215 (subscriber, #98099) [Link]

My impression is that QEMU's stance on Rust is roughly "might be interesting but unless a group of Rust experts turn up who want to do the massive amounts of work involved in demonstrating and then implementing a transition plan, it's probably not happening any time soon". Some parts that can be offloaded into separate processes that talk some defined protocol have been done in Rust, but they are all out of tree separate projects. Rust in the main QEMU binary is a much trickier thing.

Committing to Rust for kernel code

Posted Nov 22, 2023 19:05 UTC (Wed) by josh (subscriber, #17465) [Link] (2 responses)

> the language still isn't entirely stable

Clarification: Rust provides a stable version, but the Linux kernel uses nightly Rust because it wants features that aren't stable yet. This will get fixed once all the remaining nightly features used in the kernel are either stabilized or replaced with some other mechanism.

Committing to Rust for kernel code

Posted Nov 22, 2023 21:06 UTC (Wed) by Conan_Kudo (subscriber, #103240) [Link] (1 responses)

That assumes that they won't just consume more Rust nightly stuff. I don't have confidence in that.

Committing to Rust for kernel code

Posted Nov 23, 2023 1:54 UTC (Thu) by josh (subscriber, #17465) [Link]

I'm fairly confident in that, given that they've specifically said they want to get to stable Rust.

Committing to Rust for kernel code

Posted Nov 22, 2023 23:19 UTC (Wed) by ojeda (subscriber, #143370) [Link] (4 responses)

> GCC support will be a while

To add some context on the two GCC approaches:

rustc_codegen_gcc is already able to compile the kernel without source code changes (modulo unrelated alloc changes). Though still experimental, it looks like it is the first approach we will be able to use to build through GCC. See https://rust-for-linux.com/rustc_codegen_gcc for the latest status and kernel-specific instructions from Antoni Boucher (the rustc_codegen_gcc lead developer) who kindly maintains that page in our website.

GCC Rust (gccrs) is working on upstreaming all their work so far, to be released in GCC 14. They have done a lot of work, but upstreaming takes a long time and it is still a long way from compiling the kernel. However, building Rust for Linux is one of their explicit objectives. See this very recent talk from Arthur Cohen (co-lead developer of GCC Rust) or the slides in our Rust for Linux talk at LPC that Arthur generously provided.

Both approaches should allow the Rust side of the kernel to access GCC-only architectures. However, due to the nature of each approach, GCC Rust will take longer to be useful for the kernel purposes, which is why I think more resources would help them significantly.

Committing to Rust for kernel code

Posted Nov 23, 2023 15:02 UTC (Thu) by rahulsundaram (subscriber, #21946) [Link] (3 responses)

> GCC Rust will take longer to be useful for the kernel purposes, which is why I think more resources would help them significantly.

What if any do you see as an advantage to the second approach?

Committing to Rust for kernel code

Posted Nov 23, 2023 17:09 UTC (Thu) by atnot (guest, #124910) [Link] (2 responses)

So far I've seen a few:
- A second implementation to validate against. With their ambition to be bug-compatible and to share the borrow checker with rustc (probably the least rigorously specified part of rust) that does feel a bit doubtful.
- Having a fully copyleft implementation. See previous for limitations. This doesn't seem to be an issue for languages like Python, but some people do seem to care.
- Another path for bootstrapping. mrustc already covers this mostly, but it's an option.
- Possibly better cross-language builds. I haven't really seen any examples of this from other gcc frontends, but it's plausible.
- Force of habit of compiling native code with gcc.
- Desire to keep GCC relevant over LLVM, especially as a flagship for GNU.
- Vague ambitions to dethrone the rustc developers because you don't like them or they won't let you say racial slurs
- All of the people who reflexively claimed they didn't want to use rust because it doesn't have a GCC frontend will have to find a new excuse. Perhaps they'll suddenly become really big fans of ISO.

Although all of these are mostly just reasons for it existing somewhere in the aether, not for anyone actually using it as their primary rust compiler.

I don't think any of these reasons are incredibly convincing on their own, but between them they do fund a few positions. Is that enough? I don't know. But to be a bit cynically, does it really matter when it seems to be most important as a symbolic gesture for everyone involved anyway?

Committing to Rust for kernel code

Posted Nov 23, 2023 17:30 UTC (Thu) by atnot (guest, #124910) [Link]

Oh, to be clear, this is not me saying they shouldn't do it. I don't think it hurts anyone and it does seem like it could come in handy sometimes. I'm just not convinced it's something that will change the trajectory of rust in any sense or that there's actually an alleged army of fence-sitters out there who would just *love* to use rust if it weren't for rustc.

Committing to Rust for kernel code

Posted Nov 24, 2023 12:32 UTC (Fri) by ballombe (subscriber, #9523) [Link]

if the kernel is required to work with both compilers, this will slowdown the rise of the 'minimal supported version' of rustc

There have been situation where you could not build a newer release of firefox because the rust compiler provided the distro was too old, and that upgrading rust was a major update.
This is something I am sure a lot of kernel users are concerned about.

Committing to Rust for kernel code

Posted Nov 23, 2023 15:54 UTC (Thu) by atnot (guest, #124910) [Link] (2 responses)

> Williams pointed out that the previous session had discussed how hard it is to get kernel subsystems to move to new APIs; now, he said, there is talk of moving to a whole new language

I think it's worth considering why those moves are so hard instead of just taking it as a given in such a context. It's generally not that there's a lack of people with the ability to do such rote transformations. Often they can even be done automatically. The part that's difficult is building the confidence that those changes are correct, which usually requires intensive auditing of the surrounding code, manual testing and extensive domain knowledge. Those resources are scarce.

There's a common definition of Legacy Code as "Code that is too scary to change and too valuable to delete". Arguably this applies to a lot of the kernel today. Short of stopping development or going on a deletion spree, there is really no alternative to finding ways of getting more confidence in changes. That is going to involve a bit of churn, no matter the method you choose. But that investment really does pay off quickly.

Committing to Rust for kernel code

Posted Nov 27, 2023 14:14 UTC (Mon) by NYKevin (subscriber, #129325) [Link] (1 responses)

> There's a common definition of Legacy Code as "Code that is too scary to change and too valuable to delete".

At my workplace, one of the more useful rules of thumb* is that code must be understood by somebody, or else it must be removed (eventually, with appropriate care and planning). If deleting module X causes module Y to break, then you go to the owners of Y and tell them "either you take on ownership of X, or else you stop using it." If there are multiple values of Y, then they can fight it out amongst themselves.

Unfortunately, this probably works better in a corporate setting than in a FOSS setting (where nobody has a budget, a PM, OKRs, etc.). But I think it could still be an instructive analogy. Ideally, you should not have large heaps of code that nobody can read and everybody is too scared to delete. The question is, how do you get there from here?

* This is not a formal "rule" in the sense that it's a policy everybody follows. It's an informal best practice that I have heard from various high-level engineers.

Committing to Rust for kernel code

Posted Nov 27, 2023 14:24 UTC (Mon) by pizza (subscriber, #46) [Link]

> Unfortunately, this probably works better in a corporate setting than in a FOSS setting (where nobody has a budget, a PM, OKRs, etc.).

Don't forget that in the F/OSS setting you're much more likely to be (very) decoupled from the end-users of a given bit of code. Sure, you remove something today, but your general userbase is using LTS-type distros it could easily be years before they notice.

I suppose the more generalized way to put it is that this "take ownership of X" only really applies if the code in question is under active development, as opposed to simply being ignored because "it works and there's no need to mess with it"