Standardizing BPF

Posted Apr 10, 2023 21:58 UTC (Mon) by jhoblitt (subscriber, #77733)
Parent article: Standardizing BPF

I'm surprised there is so much pressure to run through a traditional standards organization. AFAIK, the wasm spec hasn't been submitted to a formal standards body and that doesn't seem to have slowed down it's adoption. I guess the stability concerns for storage device firmware are greater than for a browser which may be updated daily -- probably a good thing.

to post comments

Standardizing BPF

Posted Apr 10, 2023 22:52 UTC (Mon) by kaesaecracker (subscriber, #126447) [Link] (3 responses)

There is https://www.w3.org/TR/wasm-core-1/, I think HTML5 had the recommendation status for a long time too.

Standardizing BPF

Posted Apr 11, 2023 6:42 UTC (Tue) by NYKevin (subscriber, #129325) [Link] (1 responses)

HTML5 is weird. Prior to 2019, it was published by both W3C and WHATWG, both of which considered their version to be authoritative. WHATWG was (and still is) publishing a "living standard" that changes as the web develops, while W3C was essentially making "snapshots" of the WHATWG's work with minor changes (or at least, that is how Hixie characterized it at the time - the W3C might have disagreed with him, but I wasn't able to find their public comments on the matter). The current agreement[1] is essentially that the WHATWG will write "drafts" and then the W3C will republish them as "recommendations" (subject to a lot of bureaucracy so that disagreements can be resolved if necessary - but for the most part, disputes are resolved within the WHATWG's internal processes, and the W3C merely reserves the right to fork as a last resort).

Arguably, this was the inevitable result of a conflict between standards body and implementers - if nobody will implement your standard, then your standard is worthless paper. Unfortunately, quite a few standards bodies don't seem to have realized this, perhaps because software is uniquely amenable to rapid divergence between standards and implementations. Contrast, for example, the IETF's "rough consensus and running code" practice with, say, the ISO's army of bureaucrats.

[1]: https://www.w3.org/2019/04/WHATWG-W3C-MOU.html

Standardizing BPF

Posted Apr 11, 2023 12:36 UTC (Tue) by excors (subscriber, #95769) [Link]

HTML5 has had a rather messy history. It started in 2004 when some browser developers wanted to work on modest improvements to HTML4 forms, but the W3C voted no: the future was XHTML2. The browser developers realised they didn't need the W3C at all; its legitimacy and authority over web standards was largely a fiction, and they had enough market share that they could just write and publish their own specification and it would become the de facto standard, so they formed the WHATWG to do that.

They also had a very different culture: the W3C was quite a closed environment, with teleconferences and private mailing lists limited to W3C Member organisations and Invited Experts, and a formal consensus process, with many participants from an academic background. The WHATWG was mostly very open, based around a public mailing list and IRC channel where any random person could join and participate in discussions, regularly publishing working drafts from the public Subversion repository, and I think the most active participants were largely open-source-friendly software engineers and others with a similar background and perspective. Also the WHATWG was a benevolent dictatorship where one person (Ian Hickson) made every decision and wrote every line of specification text, and its success depended entirely on him doing a good job over many years.

The biggest problem for the WHATWG was that Microsoft was unwilling to get involved without a proper patent policy. Meanwhile the W3C (who did have a patent policy) realised that XHTML2 was not the future and they had lost control of the evolution of the web, so they reluctantly joined forces in 2007. For the WHATWG, it was seen as a necessary evil for getting Microsoft on board - there was basically no interest in the W3C Process, because most WHATWG participants much preferred their own way of working, so both groups worked in parallel on the same document under the same editor but with different discussion forums and different decision-making processes.

Perhaps unsurprisingly, the relationship between the two groups was highly acrimonious. They had some fundamentally different perspectives on technical and social matters and couldn't get along with each other, but both believed the future of the web was too important to back down. In the early years, I think that made the W3C's HTML WG a miserable place for everyone and the mailing list was full of endless flame wars and it achieved basically nothing, while most of the technical work continued in the WHATWG.

If I remember correctly, at first both groups published different documents based on the same source file: sometimes the W3C would formally make a decision that the editor strongly disagreed with, so he would include both variations in the source and a preprocessor would pick the correct text for each publication. Later the two sides drifted further apart, as the W3C focused on getting snapshots of the specification through their Recommendation process while the WHATWG continued evolving it incrementally (the Living Standard, which better matches how HTML is implemented and deployed in practice), resulting in many more incompatible forks and much confusion and unhappiness.

I stopped paying attention around that time, but it sounds like the WHATWG formalised its processes and adopted an IPR policy in 2017 (presumably so Microsoft could work with it directly), and the W3C stopped its parallel work in 2019 and now most of its old URLs simply redirect to the WHATWG Living Standard, to solve the confusion caused by forks. So it appears the WHATWG 'won' in the end, but it was a painful journey.

I'm not sure what my point was, but maybe there's a lesson like: When standardising a technology where there's a natural incentive for interoperability (so you don't need to e.g. legally mandate that people follow the standard), the name or reputation of the organisation doesn't matter - you can even make up your own organisation and succeed. The most important factor is having the right people involved, which depends largely on culture (who you'll attract to participate and how well they'll work together) and IPR policy (so their employer will allow them to participate), so that you can do a technically competent job and get buy-in from a critical mass of implementers, and then you'll have a successful standard.

Standardizing BPF

Posted Apr 11, 2023 8:38 UTC (Tue) by anselm (subscriber, #2796) [Link]

Note that with the W3C, documents don't get more official than “recommendation”. What the W3C calls a recommendation is what ISO and friends would call a standard.

Standardizing BPF

Posted Apr 17, 2023 16:44 UTC (Mon) by jontrossbach (guest, #164668) [Link]

Very interesting write-up, still have a question though. My understanding of most RFCs and how they got approved comes from Peter Dordel's book AItCN (~pg 37, 1.15 IETF and OSI). In summary, most successful RFCs are the results of an industry generally already having been standardized around a somewhat accepted standard. Attempts to generically define an RFC in a top down manner have (not always but) generally failed or gone less smoothly in the past. Dordel points out the reason OSI Layers are so sloppily defined is because it was largely a top down definition many often ignored or didn't fit squarely into.

Is there a somewhat interoperable working standard most in the industry are already working off of for eBPF or is this an attempt at something like a top down RFC definition? If the former, then is that standard largely defined anywhere yet?

Or are we just in the phase of asking the question: should this ever be an RFC at all for any reason?

Also, what are the best resources to look at for what will eventually likely be the proposed standard? I seem to be confused on that as well. I think a chart showing who is already using what would be helpful in trying to wrap ones head around the details of what is going on here, if we're mostly to a point where such a chart can be made, that is.