Broadcom's wireless drivers, one year later

By Jonathan Corbet
August 29, 2011

On September 9, 2010, Broadcom announced that it was releasing an open source driver for its wireless networking chipsets. Broadcom had long resisted calls for free drivers for this hardware, so this announcement was quite well received in the community, despite the fact that the quality of the code itself was not quite up to contemporary kernel standards. One year later, this driver is again under discussion, but the tone of the conversation has changed somewhat. After a year of work, Broadcom's driver may never see a mainline release.

Broadcom's submission was actually two drivers: brcmsmac for software-MAC chipsets, and brcmfmac for "FullMAC" chipsets with hardware MAC support. These drivers were immediately pulled into the staging tree with the understanding that there were a lot of things needing fixing before they could make the move to the mainline proper. In the following year, developers dedicated themselves to the task of cleaning up the drivers; nearly 900 changes have been merged in this time. The bulk of the changes came from Broadcom employees, but quite a few others have contributed fixes to the drivers as well.

This work has produced a driver that is free of checkpatch warnings, works on both small-endian and big-endian platforms, uses kernel libraries where appropriate, and generally looks much better than it originally did. On August 24, Broadcom developer Henry Ptasinski decided that the time had come: he posted a patch moving the Broadcom drivers into the mainline. Greg Kroah-Hartman, maintainer of the staging tree, said that he was fine with the driver moving out of staging if the networking developers agreed. Some of those developers came up with some technical issues, but it appeared that these drivers were getting close to ready to make the big move out of staging.

That was when Rafał Miłecki chimed in: "Henry: a simple question, please explain it to me, what brcmsmac does provide that b43 doesn't?" Rafał, naturally, is the maintainer of the b43 driver; b43, which has been in the mainline for years, is a driver for Broadcom chipsets developed primarily from reverse-engineered information. It has reached a point where, Rafał claims, it supports everything Broadcom's driver does with one exception (BCM4313 chipsets) that will be fixed soon. Rafał also claims that the b43 driver performs better, supports hardware that brcmsmac does not, and is, in general, a better piece of code.

So Rafał was clear on what he thought of the brcmsmac driver (brcmfmac didn't really enter into the discussion); he was also quite clear on what he would like to see happen:

We would like to have b43 supported by Broadcom. It sounds much better, I've shown you a lot of advantages of such a choice. Switching to brcmsmac on the other hand needs a lot of work and improvements.

On one hand, Rafał is in a reasonably strong position. The b43 driver is in the mainline now; there is, in general, a strong resistance to the merging of duplicate drivers for the same hardware. Code quality is, to some extent, in the eye of the beholder, but there have been few beholders who have gone on record to say that they like Broadcom's driver better. Looking at the situation with an eye purely on the quality of the kernel's code base in the long term, it is hard to make an argument that the brcmsmac driver should move into the mainline.

On the other hand, if one considers the feelings of developers and the desire to have more hardware companies supporting their products with drivers in the Linux kernel, one has to ask why Broadcom was allowed to put this driver into staging and merge hundreds of patches improving it if that driver was never going to make it into the mainline. Letting Broadcom invest that much work into its driver, then asking it to drop everything and support the reverse-engineered driver that it declined to support one year ago seems harsh. It's not a story that is likely to prove inspirational for developers in other companies who are considering trying to work more closely with the kernel community.

What seems to have happened here (according mainly to a history posted by Rafał, who is not a disinterested observer) is that, one year ago, the brcmsmac driver supported hardware that had no support in b43. Since then, b43 has gained support for that additional hardware; nobody objected to the addition of duplicated driver support at that time (as one would probably expect, given that the Broadcom driver was in staging). Rafał doesn't say whether the brcmsmac driver was helpful to him in filling out hardware support in the b43 driver. In the end, it doesn't matter; it would appear that the need for brcmsmac has passed.

One of the most important lessons for kernel developers to learn is that one should focus on the end result rather than on the merging of a specific piece of code. One can argue that Broadcom has what it wanted now: free driver support for its hardware in the mainline kernel. One could also argue that Broadcom should have worked on adding support to b43 from the beginning rather than posting a competing driver. Or, failing that, one could say that the Broadcom developers should have noticed the improvements going into b43 and thought about the implications for their own work. But none of that is going to make the Broadcom developers feel any better about how things have turned out. They might come around to working on b43, but one expects that it is not a hugely appealing alternative at the moment.

The kernel process can be quite wasteful - in terms of code and developer time lost - at times. Any kernel developer who has been in the community for a significant time will have invested significant time into code that went straight into the bit bucket at some time or other. But that doesn't mean that this waste is good or always necessary. There would be value in finding more reliable ways to warn developers when they are working on code that is unlikely to be merged. Kernel development is distributed, and there are no managers paying attention to how developers spend their time; it works well in general, but it can lead to situations where developers work at cross purposes and somebody, eventually, has to lose out.

That would appear to have happened here. In the short term, the kernel and its users have come out ahead: we have a choice of drivers for Broadcom wireless chipsets and can pick the best one to carry forward. Even Broadcom can be said to have come out ahead if it gains better support for its hardware under Linux. Whether we will pay a longer-term cost in developers who conclude that the kernel community is too hard to work with is harder to measure. But it remains true that the kernel community can, at times, be a challenging place to work.

Index entries for this article
Kernel	Broadcom 43xx
Kernel	Development model/Driver merging
Kernel	Device drivers/Wireless networking

to post comments

Broadcom's wireless drivers, one year later

Posted Aug 29, 2011 23:15 UTC (Mon) by marcH (subscriber, #57642) [Link] (6 responses)

> The kernel process can be quite wasteful

... unlike closed source developments, where millions and millions of dollars would never ever end up wasted :-)

Broadcom's wireless drivers, one year later

Posted Aug 31, 2011 22:09 UTC (Wed) by mhelsley (guest, #11324) [Link] (5 responses)

The article remains quite correct. It really doesn't matter whether closed source processes are better or worse. The real issue is whether/how the *kernel* process can/will be improved to help avoid "wasting" those resources.

Broadcom's wireless drivers, one year later

Posted Aug 31, 2011 22:19 UTC (Wed) by marcH (subscriber, #57642) [Link] (3 responses)

Breaking news: after ignoring (open) source code from closed source shops, the Linux kernel community now decides to ignore their processes as well.

http://en.wikipedia.org/wiki/Not_Invented_Here

Broadcom's wireless drivers, one year later

Posted Sep 1, 2011 7:58 UTC (Thu) by philipstorry (subscriber, #45926) [Link] (1 responses)

I refuse to use this nomenclature, as it doesn't meet my own already established patterns or integrate well with the tools I use every day.

I have created my own expression for this phenomenon, which I call Invented Somewhere Else (ISE). This has numerous advantages over the current NIH tool, but I do not have the space or time to document them here. I encourage you to try ISE, and find its benefits for yourself - they're easy to spot even without my guidance.

I hope to merge ISE into English Mainline for the 2012 release, once I've convinced its maintainers that my writing style is not idiosyncratic and obscure but is in fact superior and inspired.

Watch this space!

Adopting ISE

Posted Sep 6, 2011 7:46 UTC (Tue) by man_ls (guest, #15091) [Link]

It was hard to get (I blame the early hour and the relative lack of caffeine), but thanks for a good laugh! Self-referencing humor always cracks me up.

Broadcom's wireless drivers, one year later

Posted Sep 9, 2011 9:52 UTC (Fri) by dag- (guest, #30207) [Link]

If you ignore all the rationale, arguments and history, you might get away with blaming this to NIH. However, given how this turned out, if b43 would not be on par with Broadcom's driver, the outcome would have been different, despite it Not having been Invented Here.

And that is exactly what one would expect.

Broadcom's wireless drivers, one year later

Posted Sep 1, 2011 8:15 UTC (Thu) by mjthayer (guest, #39183) [Link]

> The article remains quite correct. It really doesn't matter whether closed source processes are better or worse. The real issue is whether/how the *kernel* process can/will be improved to help avoid "wasting" those resources.

Dare I ask, might part of the problem here be lumping too much together into one piece of software? If you look into user-space, you see healthy competition between (free) projects such as NetworkManager and ConnMan [1] which presumably benefit both and all users. I'm not saying that a situation where every driver lives for itself, as is customary in other OSes (or the old situation where every wireless driver on Linux included its own IEEE 802.11 stack), would be an improvement, but if say competing complete wireless subsystems complete with drivers could co-exist without one of them having to be the single chosen one, we might be able to cut out some of the time wasted on politics. No, I didn't say the MK word.

[1] http://lwn.net/Articles/456967/

Broadcom's wireless drivers, one year later

Posted Aug 29, 2011 23:56 UTC (Mon) by PaulWay (guest, #45600) [Link] (2 responses)

Forgive this for being somewhat blunt, but how did this go on for eleven months without anyone pointing out that the Broadcom developers were aiming for a twin of b43? Why didn't someone back in September 2010 say to them "well, that's nice and all, and we can help you get that ready for inclusion into the kernel, but we already have this perfectly good driver, why don't you save yourself some work and just improve that?" Why didn't Rafał say something at that stage, rather than eleven months down the track?

It's obvious, perhaps, in hindsight. I'm sure there are reasons that these delays occurred. And I can understand from Broadcom's perspective it may still think that it needs to support its own code rather than someone else's. But in general we've been asking, begging, and extending olive branches to hardware manufacturers; we really need to make these interactions work. If more manufacturers thought they could piggy-back on the existing work of the Linux kernel and help out rather than having to write everything from scratch and then go through the arduous process of preparing it for mainline only to meet resistance from existing driver maintainers.

Have fun,

Paul

Broadcom's wireless drivers, one year later

Posted Aug 30, 2011 0:54 UTC (Tue) by dlang (guest, #313) [Link] (1 responses)

the key is that a year ago the b43 driver didn't support a lot of hardware that the broadcom driver already supported, and there was little (if anything) that the b43 driver supported that the broadcom driver didn't support

so it was easy to see the broadcom driver as being a superset of the b43 driver.

they probably thought that it would be less work to modify the already working broadcom driver to fit the kernel better than it would be to modify the reverse engineered b43 driver to support all the other stuff.

as it appears to have turned out, the team working to improve the b43 driver was able to add support faster than the broadcom team was able to revamp their driver to fit the kernel.

it could have easily turned out the other way. If the changes had not turned out to be as intrusive, or if the b43 team had not been as quick to add new support we could easily be celebrating replacing the reverse engineered driver that works for most people most of the time with a driver that supports all the chipsets all of the time because the people developing it understand the chipsets better.

and there is still the unanswered question of how much the broadcom driver being in staging helped the b43 driver team

Broadcom's wireless drivers, one year later

Posted Aug 30, 2011 17:44 UTC (Tue) by mlankhorst (subscriber, #52260) [Link]

To be fair, if you read the thread you do see there are things that just work better with brcmsmac driver, especially 802.11n, but it lacks features that b43 has, like using it on a SoC router as AP. Especially the PHY code might be useful for b43.

Disclaimer: I'm not a wireless-dev :)

Broadcom's wireless drivers, one year later

Posted Aug 30, 2011 0:17 UTC (Tue) by tobiasu (subscriber, #72521) [Link]

All because Broadcom doesn't publish proper datasheets and let the community do the implementation. I pity them - not.

Broadcom's wireless drivers, one year later

Posted Aug 30, 2011 0:31 UTC (Tue) by GrueMaster (guest, #57504) [Link] (2 responses)

Didn't we go through this back in kernel 2.2 with the Intel Ethernet drivers? Seems to me that we had some level of duplication on the 100Mb driver that was eventually resolved.

Broadcom's wireless drivers, one year later

Posted Aug 30, 2011 1:21 UTC (Tue) by josh (subscriber, #17465) [Link]

And more recently with the nForce driver for nVidia Ethernet, and the reverse-engineered "forcedeth" driver. In that case, nVidia deprecated their own driver in favor of forcedeth.

Broadcom's wireless drivers, one year later

Posted Aug 30, 2011 6:11 UTC (Tue) by Lumag (subscriber, #22579) [Link]

Also we still have aic7xxx and aic7xxx_old SCSI drivers.

Broadcom's wireless drivers, one year later

Posted Aug 30, 2011 9:34 UTC (Tue) by dpc (guest, #74012) [Link] (4 responses)

That's the way of the Bazaar.

Broadcom's wireless drivers, one year later

Posted Aug 31, 2011 22:26 UTC (Wed) by arget (guest, #5929) [Link] (3 responses)

Exactly. I don't see any "waste" here. Is it a waste of effort for two stores to sell shoes across the street from each other? One can look in at each window, check out the styles on display, try on a pair in each. One store will make a sale, one won't. Competition is not waste, it drives innovation, inspires creativity, and motivates teams to push just a little bit harder.

Broadcom's wireless drivers, one year later

Posted Aug 31, 2011 23:16 UTC (Wed) by mwsealey (subscriber, #71282) [Link] (2 responses)

This is more like having two Starbucks stores next door to each other and the only differentiating feature between them is that you like the baristas better at Starbucks #1 but there's this guy who stinks of cheese who's always torrenting on the free wifi. Starbucks #2 has so-so baristas but no cheesey pirate so you can get your free iTunes song while they froth your milk.

What we really need to focus on here is that at the end of the day.. it's coffee..

Broadcom's wireless drivers, one year later

Posted Sep 1, 2011 3:04 UTC (Thu) by arget (guest, #5929) [Link]

Sadly, I think I know a corner with two Starbucks exactly like this.

Broadcom's wireless drivers, one year later

Posted Sep 1, 2011 9:25 UTC (Thu) by spaetz (guest, #32870) [Link]

> What we really need to focus on here is that at the end of the day.. it's coffee..

You do recognize that people kill for *the right* way to prepare coffee? :)

Yahoo: 126m hits for "coffee war"
http://www.coffeeco.com.au/altcof/altcoffeepage13.html (How to be a coffee fanatic)
http://cafefanatic.com/
http://coffeefanaticsguide.com

Broadcom's wireless drivers, one year later

Posted Aug 30, 2011 10:02 UTC (Tue) by giggls (subscriber, #48434) [Link]

We had a simular situation in the past with eepro100 and e100 drivers didn't we?

Broadcom's wireless drivers, one year later

Posted Aug 30, 2011 10:44 UTC (Tue) by gmartres (guest, #74834) [Link]

Note that the driver in mainline is not perfect yet, according to the ml thread it doesn't support 802.11n yet (but it does support some stuff the driver in staging doesn't).

This is entirely unexpected.

Posted Aug 30, 2011 11:31 UTC (Tue) by dwmw2 (subscriber, #2063) [Link] (1 responses)

"Letting Broadcom invest that much work into its driver, then asking it to drop everything and support the reverse-engineered driver that it declined to support one year ago seems harsh. It's not a story that is likely to prove inspirational for developers in other companies who are considering trying to work more closely with the kernel community."

That's an interesting take on it. Broadcom were told, a year ago, that it was stupid and wrong to introduce a new driver, and that they should instead focus on improving b43. Not that they should have needed telling.

Broadcom are in this position precisely because they were not "trying to work more closely with the kernel community".

And they're not even keeping up with new hardware. We now have things like BCM4331 being supported by b43 first, with no sign of support for it turning up in brcmsmac any time soon. The brcmsmac driver was always a dead end; the only potential benefit was that it might support new hardware faster, and even that turns out not to be the case.

We could probably salvage some of its PHY code it would be useful to make the PHY sub-drivers compatible between b43 and brcmsmac so they can be shared, perhaps. But the rest of it is just pointless, as far as I can tell.

This is entirely unexpected.

Posted Sep 1, 2011 8:30 UTC (Thu) by mjthayer (guest, #39183) [Link]

> We could probably salvage some of its PHY code it would be useful to make the PHY sub-drivers compatible between b43 and brcmsmac so they can be shared, perhaps. But the rest of it is just pointless, as far as I can tell.

Actually I was thinking that an elegant solution to this might be more clear acknowledgement in B43 of how Broadcom's work has helped them - perhaps along with an attempt to explicitly re-use more of their code. That would make it clearer to Broadcom that their work was not for naught.

Worth a read

Posted Aug 30, 2011 15:59 UTC (Tue) by corbet (editor, #1) [Link] (29 responses)

Here's a response from Henry Ptasinski that hadn't been posted when the article was written; he describes the advantages he sees in the brcmsmac driver over b43. What's going to hurt, though, is this statement:

The brcmsmac driver has architectural alignment with our drivers for other operating systems, and we intend to to enhance and maintain this driver in parallel with drivers for other operating systems. Maintaining alignment between our Linux driver and drivers for other operating systems allows us to leverage feature and chip support across all platforms.

That is not an approach that flies well in the kernel community; Linux drivers should be Linux drivers. This "architectural alignment" can only be maintained for as long as Broadcom keeps control over the driver, and control is one thing that is hard to have in the free software world.

Worth a read

Posted Aug 30, 2011 17:06 UTC (Tue) by rahvin (guest, #16953) [Link] (6 responses)

I'd wager that comment alone killed any chance of the driver ever being accepted. As you said, similar comments by other companies in the past got their code blackballed.

Cross-platform drivers

Posted Aug 31, 2011 8:30 UTC (Wed) by mjthayer (guest, #39183) [Link] (5 responses)

> I'd wager that comment alone killed any chance of the driver ever being accepted. As you said, similar comments by other companies in the past got their code blackballed.

Warning: the following complaining is going to be slightly off-topic. This brings back memories of another recent LWN article [1]. And it makes me feel, once again, that the kernel community are a bit too unforgiving or inflexible in their insistence on things being done exactly their way. Of course I understand and sympathise with their arguments that they can avoid lots of unnecessary duplication between drivers for different hardware, as avoiding duplication generally raises software quality (while lowering costs). But sharing driver code between different operating systems is also avoiding duplication, and I think that if the kernel community wanted to they could help vendors produce drivers which go some way towards achieving both. Or is there a feeling that this is giving unneeded (since the community has the resources to manage without vendor drivers) help to the competition?

I must note that I am somewhat biased as I work among other things on the cross-platform driver code of the VirtualBox Guest Additions (which we are not likely to try to get upstream in the foreseeable future).

[1] http://lwn.net/Articles/454716/

Cross-platform drivers

Posted Aug 31, 2011 12:19 UTC (Wed) by vonbrand (subscriber, #4458) [Link] (3 responses)

You just have to consider that the work load considered by kernel hackers is their burden, if lightening theirs means others have to work more, so be it.

Cross-platform drivers

Posted Aug 31, 2011 13:01 UTC (Wed) by mjthayer (guest, #39183) [Link] (2 responses)

> You just have to consider that the work load considered by kernel hackers is their burden, if lightening theirs means others have to work more, so be it.

Certainly, but does sharing code (in a sensible way of course) with other platforms make that burden heavier?

Cross-platform drivers

Posted Sep 1, 2011 21:08 UTC (Thu) by mlankhorst (subscriber, #52260) [Link] (1 responses)

I'd like to point you to an article 2 weeks ago in lwn

https://lwn.net/Articles/454716

Cross-platform drivers

Posted Sep 2, 2011 8:12 UTC (Fri) by mjthayer (guest, #39183) [Link]

> I'd like to point you to an article 2 weeks ago in lwn

I actually quoted that myself a couple of posts up this thread!

Cross-platform drivers

Posted Aug 31, 2011 15:54 UTC (Wed) by bronson (guest, #4806) [Link]

I haven't seen Linux developers worry about helping the competition (with one notable Reiser exception). This might be partly GPLv2 smugness ("go ahead, I'll just poach your changes when you release them") but I think it's just focusing on what's important. Rather than crying about calling something Linux/Android or aligning marketing messages, kernel devs just keep improving their piece of the universe.

As for the Broadcom driver, who wants to maintain another 4 files of useless shims? Or merge a do-nothing patch that contains changes needed by the Win32 ARM driver? Or try to navigate a mess of invisible dependencies known only to Broadcom devs? That would be madness.

So, I think the reservations of the kernel devs are 100% practical. They're just maximizing the maintainability of the Linux driver.

If Broadcom wants to release the source to all their other drivers too, then I expect the Linux kernel crew would be willing to talk about code duplication. Until then, they can only work with what they know.

Worth a read

Posted Aug 31, 2011 9:48 UTC (Wed) by fb (guest, #53265) [Link] (18 responses)

> > Maintaining alignment between our Linux driver and drivers for other operating systems allows us to leverage feature and chip support across all platforms.

Sounds like a very reasonable argument from my point of view.

> That is not an approach that flies well in the kernel community; Linux drivers should be Linux drivers.

Oh, but you see, sir, "Linux drivers should be Linux drivers", we only care about duplication inside the kernel itself. We actually encourage, or should I say, demand duplication in a vendor's own software projects.

In a year from now, I bet we'll have 'another LWN kernel conference converage' where Linux devs complain that vendors don't try to work along with them.

Worth a read

Posted Aug 31, 2011 10:55 UTC (Wed) by marcH (subscriber, #57642) [Link]

> Oh, but you see, sir, "Linux drivers should be Linux drivers", we only care about duplication inside the kernel itself. We actually encourage, or should I say, demand duplication in a vendor's own software projects.

Thanks for the quote of the week.

Worth a read

Posted Aug 31, 2011 11:34 UTC (Wed) by patrick_g (subscriber, #44470) [Link] (8 responses)

>>>Sounds like a very reasonable argument from my point of view.

Have you seen this mail from Greg KH in the thread? IMHO he asked very good questions and the "reasonable argument" made by Broadcom devs was destroyed.

Worth a read

Posted Aug 31, 2011 12:36 UTC (Wed) by marcH (subscriber, #57642) [Link]

> the "reasonable argument" made by Broadcom devs was destroyed.

... and they were so "destroyed" that they did not even answer.

More seriously, I suspect Greg's GPL comment is off-topic: "architectural alignment" does not sound like sharing code.

Greg's maintenance concern is likely to be moot as well. As an adaptation layer a driver like this one has two "sides": a low-level side interfacing with the hardware and, a high-level side interfacing with the operating system. Now guess which side Broadcom wants to keep "architecturally aligned" across operating systems?

Worth a read

Posted Aug 31, 2011 13:20 UTC (Wed) by fb (guest, #53265) [Link] (2 responses)

> Have you seen this mail from Greg KH in the thread? IMHO he asked very good questions and the "reasonable argument" made by Broadcom devs was destroyed.

IMHO the Broadcom devs argument remains reasonable.

Collaboration needs to happen both ways. They are willing to provide a driver, clean it and maintain it, and I find it reasonable for the kernel to takes steps to make that practical for them as well. Keep in mind that the code released by Broadcom lead to improvements to the kernel's own b43 driver.

Let them maintain that driver alone, or make outside contributions, say, Apache/GPL licensed. (Isn't it the case that when Linux took code from a BSD kernel, people decided that it should remain dual licensed?) GKH implicitly assumes that a dual-license for their OS independent code is impossible. That is precisely we demand code duplication at your company's side.

Kernel devs asking companies (managers & coders) to work along with them should understand that these folks also have other constraints in life than the Linux kernel.

Worth a read

Posted Aug 31, 2011 14:20 UTC (Wed) by corbet (editor, #1) [Link] (1 responses)

The improvements to the b43 driver are unproven at this point. They may well exist, but I would not just assume that they exist.

You're suggesting that we take drivers into the tree under the condition that only the vendors can modify them? We've seen that idea in the past; it doesn't work well at all. If it's free software, it needs to be free, not under vendor control. And that's before you consider that vendors always lose interest in drivers long before customers stop using the hardware. Remember that kernel developers need to think about what it's going to be like to maintain the code five or ten years into the future.

Sounds like I need to write another article :)

Worth a read

Posted Aug 31, 2011 15:50 UTC (Wed) by fb (guest, #53265) [Link]

I had understood that b43 (1) had been improved (2) by making use of the Broadcom driver, now I see that I need to brush my reading comprehension... say:

> Rafal doesn't say whether the brcmsmac driver was helpful to him in filling out hardware support in the b43 driver.

[...]

What I was suggesting though is to have (non-Broadcom) kernel devs making their contributions, say, Apache / GPL on this driver (instead of requiring Broadcom to duplicate their own stack), or (indeed) just let them maintain that themselves. Of course that only goes for as long as Broadcom is answering promptly to issues.

> Sounds like I need to write another article :)

Please do, your articles are always remarkably informative (even when I don't read them properly). However some themes get repeated often at LWN, like: kernel devs at conference XYZ ask hardware vendors to work closer to the community instead of just dumping crappy code on an FTP server.

Worth a read

Posted Sep 1, 2011 12:30 UTC (Thu) by SLi (subscriber, #53131) [Link] (3 responses)

Argh, I had missed that again, and I blame solely the way LWN links to the mails on LKML.

Why on earth is it that

1) Every link to a mail on LKML on LWN shows that mail as the root of the thread

2) The threads seem to always be otherwise incomplete in a number of different ways

I believe there are good web frontends to LKML. Why always link on one that is so horribly broken?

Worth a read

Posted Sep 1, 2011 15:30 UTC (Thu) by nix (subscriber, #2304) [Link]

The first symptom is just the way gmane works: if you want to see the whole thread, click on the subject line. The threads are very rarely incomplete in my experience: gmane has good connectivity.

A lot of people likely read l-k via gmane, and if it was receiving only a partial feed, people would already have complained. It's probably your setup. (What web browser are you using?)

Gmane

Posted Sep 1, 2011 15:43 UTC (Thu) by corbet (editor, #1) [Link] (1 responses)

We link to gmane for one very simple reason: it can be done in an automated way. If we had to fish out links manually for every message we post, it simply would not happen at all. The gmane interface is a little funky at times, but it works well enough once you get the hang of it.

Gmane

Posted Sep 1, 2011 16:44 UTC (Thu) by rahulsundaram (subscriber, #21946) [Link]

Would be nice to see the LWN codebase become Free software. This is the sort of minor things that others could reuse in various places.

Worth a read

Posted Aug 31, 2011 17:18 UTC (Wed) by daglwn (guest, #65432) [Link] (7 responses)

> In a year from now, I bet we'll have 'another LWN kernel conference
> converage' where Linux devs complain that vendors don't try to work along
> with them.

Right on. Maybe merging the Broadcom driver is not the right thing to do but dismissing it out of hand like this is exactly the wrong attitude to take.

In the end, the Linux kernel needs the vendors more than the vendors need the Linux kernel. Reasonable engineering discussion and decision is productive. Ideological intransigence is not.

Worth a read

Posted Aug 31, 2011 18:27 UTC (Wed) by marcH (subscriber, #57642) [Link] (6 responses)

> In the end, the Linux kernel needs the vendors more than the vendors need the Linux kernel.

This used to be true. If it still were, Broadcom would not even be here with brcmsmac. There are market reasons why they changed attitude.

Worth a read

Posted Aug 31, 2011 20:38 UTC (Wed) by TommyAtkins (guest, #79496) [Link] (5 responses)

There are still lots of Broadcom chips which don't have open source drivers, and they are not unusual in this. The kernel may have considerable sway in the market for PC chips, but this is not the case in the embedded market. Also, if the kernel development model becomes too painful, chip vendors can regain control by moving code into the device firmware.

Worth a read

Posted Aug 31, 2011 21:55 UTC (Wed) by marcH (subscriber, #57642) [Link] (4 responses)

> The kernel may have considerable sway in the market for PC chips, but this is not the case in the embedded market.

The embedded market is rather the "embedded marketS", and some are no different from the server market in this respect. Example: how many ARM platforms not supporting Linux?

> Also, if the kernel development model becomes too painful, chip vendors can regain control by moving code into the device firmware.

I doubt this would happen. Running more firmware usually requires more silicon, which costs money on a per-chip basis. As seen in this discussion Linux development can be costly too but since it is software it's a once off.

Worth a read

Posted Aug 31, 2011 22:14 UTC (Wed) by TommyAtkins (guest, #79496) [Link] (3 responses)

> how many ARM platforms not supporting Linux?

How would anyone other than ARM know that?

> Running more firmware usually requires more silicon, which costs money on a per-chip basis. As seen in this discussion Linux development can be costly too but since it is software it's a once off.

That trade-off only works in linux's favour if you are paying for the core which runs your firmware, but not the one running linux. If you are SoC vendor, it's the same either way. Or even in favour of *not* putting the code in linux, under some circumstances...

Worth a read

Posted Aug 31, 2011 22:34 UTC (Wed) by marcH (subscriber, #57642) [Link]

> How would anyone other than ARM know that?

I am sure you can make a good guess.

> That trade-off only works in linux's favour if you are paying for the core which runs your firmware, but not the one running linux. If you are SoC vendor, it's the same either way.

IP blocks making a SoC are designed by different teams, sometimes even different companies, each trying to minimize its corresponding cost. And the cost of running a slightly more complicated driver on the main CPU is usually zero.

If you look in history I think you will see that the cases where hardware takes more features on board are only driven by performance concerns.

PS: A large number of chip designers do not care a bit about any hardware/software trade-off. If they can push stuff to software to save money they will do it.

Worth a read

Posted Aug 31, 2011 23:21 UTC (Wed) by Julie (guest, #66693) [Link] (1 responses)

How would anyone other than ARM know that?

Would ARM actually know this? Don't they just sell IP cores to SoC developers/vendors? I thought kernel contributions were the Soc/platform vendors' responsibility after that. Isn't that part of the problem with code duplication - that they don't like to admit that rival vendors exist (or at least their PR departments don't)?

(I'm sure that there was a Thomas Gleixner article about this not too long ago but I can't find it now in the archives or remember the heading - if anyone has the link that would be great.)

Worth a read

Posted Sep 9, 2011 14:44 UTC (Fri) by wookey (guest, #5501) [Link]

Quite. Arm sell designs, 2nd parties use those to make chip designs, and 3rd parties then use _those_ to make platforms. So ARM itself has no real handle on how many actual board designs/platforms there are out there or what software is running on them. It does get data on chip shipments so royalties can be paid, but that's the closest it gets to knowing what's being done.

And yes - this diversity and the independence of downstream companies/devs is a big part of the reason for the code duplication in the kernel. Organisations like Linaro are doing their best to re-centralise/co-ordinate and share the actual development precisely because the downstream widget-makers and SOC vendors have done such a poor job of upstreaming and not NIHing. (I quite understand why - I use to be a downstream engineer and I failed miserably to get most of my code into mainline too, even though I know perfectly well why it's a good idea).

Worth a read

Posted Aug 31, 2011 11:05 UTC (Wed) by marcH (subscriber, #57642) [Link] (2 responses)

> ... and control is one thing that is hard to have in the free software world.

For good and bad, the development of Linux drivers is MUCH more controlled than for other operating systems. Not an opinion, just a fact.

Worth a read

Posted Aug 31, 2011 13:05 UTC (Wed) by tzafrir (subscriber, #11501) [Link] (1 responses)

Controlled by whom?

Who controls the Broadcom drivers for other operating systems?

Worth a read

Posted Aug 31, 2011 13:27 UTC (Wed) by marcH (subscriber, #57642) [Link]

> Controlled by whom?

Definitely a key question. Control is felt only when you do not have it.

Broadcom's wireless drivers, one year later

Posted Aug 30, 2011 19:29 UTC (Tue) by johannbg (guest, #65743) [Link] (7 responses)

Since it makes absolutely no sense having two drivers that do essentially the same thing, the best bits from both drivers should be merged together under new name ( both parties loose something ) and Broadcom requested to provided proper documentation in the future, Both parties join hands in maintaining it with the end result being in the long run an win win situation for everybody...

Broadcom's wireless drivers, one year later

Posted Aug 30, 2011 21:13 UTC (Tue) by tdwebste (guest, #18154) [Link] (6 responses)

I think this is the best way forward.

"Since it makes absolutely no sense having two drivers that do essentially the same thing, the best bits from both drivers should be merged together under new name ( both parties loose something )"

Broadcom's wireless drivers, one year later

Posted Aug 30, 2011 21:17 UTC (Tue) by dlang (guest, #313) [Link] (5 responses)

and who is going to do this work?

this isn't just a car where you can bolt in a different engine with some adapters, much of the useful meat of each driver is useless without the rest of the structure of that driver.

this completely ignores the broadcom 'we want to keep this driver structured this way so that it's like the drivers for other operating systems' statement.

Broadcom's wireless drivers, one year later

Posted Aug 30, 2011 21:40 UTC (Tue) by Julie (guest, #66693) [Link]

Also, aren't people now used to using the b43 name? I had a machine with this hardware and I know I was (usually accompanied by some swearing as I recall at the time :-/ ...)

Broadcom's wireless drivers, one year later

Posted Aug 31, 2011 18:36 UTC (Wed) by johannbg (guest, #65743) [Link] (3 responses)

>and who is going to do this work?

I would assume both parties that are trying to reach the same goal if not I would think the vendor him self would have the final saying in it after all he knows his product best and what's coming up next and has all the required information about his product and so fourth and so on hence having the vendor backing the driver development beats the reverse engineering any time. ( ofcourse the best end result is if they could work together ).

So I quote Henry's response to the thread

"As we add new, much more complicated chips, those chips will also get complete, full-performance, fully-tested support."

If you put this into end user perspective this is not even an question of which one should be dropped if it comes down to it.
( which could be limited to "once you provide equal or better support than b43 we will replace your driver with b43 in mainline" )

Fully supported Broadcom chipsets backed tested and production ready which is on various range of hw available at the same or similar time as it is for other OS vs reversed engineered Broadcom chipset limited to what ever hw range the engineers have access to and what time they have to spare working on reverse engineering it *from* the time they acquired that hw.

On in on I think the ramification of turning down vendors when they finally decide/have been talked into investing their resources into open sourcing their drivers and get it into mainline is rather great.

For all we know they might decide that this was a lost cause, flag the effort as tried and tested with expenses down the drain and never look back.

Broadcom's wireless drivers, one year later

Posted Aug 31, 2011 21:33 UTC (Wed) by raven667 (subscriber, #5198) [Link] (2 responses)

The thing that the kernel developers are optimizing for is the ability to do long-term maintenance on their own. They presume that the person who wrote the code might not be the only person who needs to maintain the code which is why they are so insistent that any code follows the style guidelines and does not have it's own internal abstraction layers or API that other kernel developers would be unfamiliar with. They are absolutely willing to pass on a working driver if it is seen to increase the long-term maintenance burden of the kernel. Given that others will end up maintaining this driver in the future, don't they have a choice in what they accept?

I think this attitude is largely driven by the reiserfs debacle, which may have been mentioned elsewhere, where a large body of code was accepted in good faith that it would be maintained but was quickly abandoned which left no-one available who understood it and was qualified to maintain it. It put the kernel developers in an awful position, distributing code that they were unable to effectively support, that they are desperate to never repeat.

Broadcom's wireless drivers, one year later

Posted Sep 1, 2011 10:02 UTC (Thu) by johannbg (guest, #65743) [Link] (1 responses)

How easy is it for vendors to access documentation on how they should interact with kernel so their side of things work as are expected?

and at the same time...

How easy is it for hackers to access vendor documentation so they can properly work on long time supporting their product?

Hence is that not something that needs to be solved with documentation requirement before acceptance?

I'm not seeing how it can be expected that somebody can provide long time support for something if he does not have the documentation on how things are supposed/expected to work.

I think both vendor and the kernel community might needs some work on providing/improving their documentation.

Broadcom's wireless drivers, one year later

Posted Sep 1, 2011 17:58 UTC (Thu) by raven667 (subscriber, #5198) [Link]

There is much documentation in the kernel in addition to information on the internet and books that are written every year or so. I think though that there is a fundamental misunderstanding that there is a separation between the kernel and vendors, the kernel is a monolithic project, intentionally without a published internal API that a third party could develop for. One is either a kernel developer or not, regardless of whether they work for one vendor or another or are a volunteer and when you are a kernel developer than you are one small part of a much larger project.

It is great if one can provide long term support for the code they write but there is a presumption that people will move on and that someone else has to be able to fill those shoes. It would be great if hardware makers would allow kernel developers who are not in their employ to see hardware specs and docs it is clearly not a hard requirement otherwise b43 wouldn't exist. There is a long history of drivers being developed without the manufacturers involvement and while the process will be slower and more error prone it is not a show stopper.

As I said before, the kernel developers (which could include Broadcom employees) would rather not have a driver with baggage attached than try and support something out side of their support model. There is nothing stoping Broadcom from being kernel developers, it's a non-exclusive club that many many hardware vendors participate in every day.

In this specific case I hope that Broadcom can convince others that the architecture of their driver is better, not only that it supports more hardware. Certainly they will not be successful in arguing that the architecture of their driver is better because it matches that of some other OS that the Linux kernel developers couldn't give a rats ass about.

Broadcom's wireless drivers, one year later

Posted Aug 30, 2011 19:30 UTC (Tue) by linusw (subscriber, #40300) [Link] (1 responses)

And then we have the acx111-mac80211 driver which is really useful for me atleast but being maintained out-of-tree:

http://acx100.git.sourceforge.net/git/gitweb.cgi?p=acx100...

Has anyone heard of a plan for that thing?

Broadcom's wireless drivers, one year later

Posted Sep 1, 2011 12:06 UTC (Thu) by cathectic (guest, #40543) [Link]

I thought acx100 was still bogged down by the fact that it was 'tainted' as it was not a clean-room reverse engineering of a binary driver?

http://sourceforge.net/mailarchive/message.php?msg_id=200...

I'm yet to see any response to that (either TI giving a thumbs up, or a new, clean-room driver produced).

Broadcom's wireless drivers, one year later

Posted Aug 30, 2011 21:33 UTC (Tue) by Julie (guest, #66693) [Link] (2 responses)

there are no managers paying attention to how developers spend their time

A sort of PA/receptionist for the busiest maintainers would be nice to help avoid this type of mix-up, and one for linux-kernel to sit on the mailing list and prod new testers/patch submitters in the right direction (maybe sponsored by the Linux Foundation? :-)

The kernel must be pretty unique for a project of its size to have _no-one_ responsible for co-ordination between its contributors.

I suppose one _might_ point a finger at the maintainer, but Greg seems to be spread waaaay too thinly as it is to have done anything about this :-/ ...

That said, I feel really sorry for the Broadcom devs, they've done a huge amount of work trying to bring the drivers up to quality and their contributions to the linuxdriverproject list have generally been a model of patience and politeness. It's a real shame if all that goes to waste.

Broadcom's wireless drivers, one year later

Posted Aug 31, 2011 18:47 UTC (Wed) by gregkh (subscriber, #8) [Link] (1 responses)

> I suppose one _might_ point a finger at the maintainer,
> but Greg seems to be spread waaaay too thinly as it is
> to have done anything about this :-/ ...

Um, what did I do wrong here? I watch the staging drivers, I don't,
and will not, track all of the wireless drivers at the same time, that's
an impossible task for anyone to assume of anyone.

As for me being "spread waaay too thinly" am I not doing something I am
supposed to be doing here? What am I doing not as well as I should?

Also note that I have a day job as well, my staging work is almost always
on my own time.

As for an "assistant" for us, sure, sign up, we'll gladly accept such
help, no one has ever offered to do that before.

Broadcom's wireless drivers, one year later

Posted Aug 31, 2011 22:34 UTC (Wed) by Julie (guest, #66693) [Link]

I'm very sorry, Greg, I think you misunderstood my comment (or maybe I wasn't clear enough). I was totally not sniping at you - quite the opposite. By 'spread waaay too thinly' I merely meant (from all I had observed) that you were already insanely busy/oversubscribed so it was unreasonable for you to take on any more work (I was specifically thinking of the wireless devs and their separate mailing list when I made this comment, the drivers project is really huge as it is. And before anyone says anything - no, I don't think it's the wireless devs' fault either).

Maintainers have to take a loose responsibility for those working in their area and they have enough to do as there is, I meant by my comment that it was a bit much to expect you all to do _everything_ (the Broadcom drivers obviously being staging ones and thus falling under your jurisdiction). From the point of view of someone observing the general kernel development process, the sensible person to look to to guide inter-dev co-ordination and avoid communication failures otherwise might be a maintainer. Maybe this seems flippant to an established kernel hacker. I didn't intend it to be. I was trying to look at it from the perspective of an (outside) company wondering where something went wrong.

As for the 'assistant' I'd love a job like this but I have to do my day job too :-) As I still learn the best I can do in the short term - which I have done and will continue to do - is prod appropriate people from kernelnewbies and other mailing lists I follow towards the linuxdriverproject list, or tell people about patch preparation and development policy (for those too lazy or confused to read Documentation/ or use google) in the hopes of making existing developers' lives a bit easier :-)
Hopefully I will become more useful in the long run :-)

Broadcom's wireless drivers, one year later

Posted Aug 31, 2011 17:29 UTC (Wed) by jezuch (subscriber, #52988) [Link] (6 responses)

> it supports everything Broadcom's driver does with one exception (BCM4313 chipsets)

Last I checked my work laptop's BCM4322 worked only with the proprietary driver; the laptop is several years old and the b43 driver has always reported "Unsupported PHY" and I always ignored the problem (don't need wireless that much). The new Broadcom driver *should* support this chip (as per the changelog of my distro's kernel) but I haven't had time to check it before going on vacation last week.

But if Rafał is right and b43 indeed supports this chip, I'll be pleasantly surprised. But it may be that I'm wrongly assuming that BCM4313 and BCM4322 and different chipsets :)

Broadcom's wireless drivers, one year later

Posted Aug 31, 2011 17:56 UTC (Wed) by mjg59 (subscriber, #23239) [Link] (5 responses)

If b43 got far enough to say "unknown phy" then it's unlikely that the Broadcom driver will work. It sounds like an SSB chip, whereas brcmsmac only handles BCMA ones.

Understanding the difference between the two drivers depends on understanding the setup of Broadcom's hardware. Each chip contains its own internal bus plus some glue to the host bus. On PCs, that'll typically be to PCI or PCIe. The internal bus may then be either SSB (for older chips) or BCMA (for newer chips). b43 doesn't include the code for handling these buses - instead, that's in separate ssb and bcma layers. Given that similar phys can be present on both SSB and BCMA hardware, that simplifies the programming - b43 simply binds to the phy, no matter what the chip's bus architecture.

I've no idea if it's true, but a commonly cited reason for brcmsmac not supporting older hardware is that the older hardware is unable to run firmware that can prevent the OS from programming arbitrary frequencies, and so FCC paranoia comes into play. In any case, brcmsmac doesn't support any ssb chips and there's no indication from Broadcom that it ever will. We could keep b43 for ssb devices and leave bcma devices to brcmsmac, but that would result in some duplication of phy code. We could merge ssb support into brcmsmac, but that would cause the driver to diverge from Broadcom's internal codebase and may well make it harder for them to merge code back and forth. We could deprecate brcmsmac and devote all development efforts to b43, but there's little sign that Broadcom would have any interest in that.

All of these solutions suck, but at some point Broadcom have to take some responsibility. If they'd been more cooperative 7 years ago, b43 wouldn't exist at all.

Broadcom's wireless drivers, one year later

Posted Sep 1, 2011 2:15 UTC (Thu) by paragw (guest, #45306) [Link] (2 responses)

"We could keep b43 for ssb devices and leave bcma devices to brcmsmac, but that would result in some duplication of phy code."

That to me sounds like the only practical option - even attractive one given the others. If we are not talking tens of thousands of LOC - I don't see why that it would be so bad.

May be some portion of "baked" phy code can be made common and shared across b43 and brcmsmac - the one that is totally obvious and is not likely to change much.

Broadcom's wireless drivers, one year later

Posted Sep 1, 2011 23:48 UTC (Thu) by mb (subscriber, #50428) [Link] (1 responses)

> That to me sounds like the only practical option - even attractive one given the others. If we are not talking tens of thousands of LOC - I don't see why that it would be so bad.

What about embedded BCMA? brcmsmac doesn't support it.

Broadcom's wireless drivers, one year later

Posted Sep 2, 2011 3:10 UTC (Fri) by paragw (guest, #45306) [Link]

>What about embedded BCMA? brcmsmac doesn't support it.

That's drivers/bcma right? Additional 1727 lines of code on top of b43+ssb - which isn't all that bad. May be getting rid of duplicate hardware support from b43 could compensate some.

Broadcom's wireless drivers, one year later

Posted Sep 10, 2011 9:46 UTC (Sat) by slashdot (guest, #22014) [Link] (1 responses)

There's always to option to leave things as they are now.

bcsmsmac stays in staging forever, and gets worked on by Broadcom engineers.

End-users mostly use b43 instead, and the b43 developers port any new feature appearing in bcsmsmac to b43 (with acknowledgements, preferably).

Broadcom's wireless drivers, one year later

Posted Sep 11, 2011 0:36 UTC (Sun) by butlerm (subscriber, #13312) [Link]

I agree. I don't see why Broadcom doesn't develop an open reference driver on an out of tree basis, according to whatever conventions they prefer, and let developers of compatible drivers use it as a reference.

That way they maintain 100% control, one code base that they can deploy across multiple operating systems, and others can do whatever they feel is appropriate for local standards, code sharing, and long term maintenance.