[go: up one dir, main page]
|
|
Log in / Subscribe / Register

Pointless ideology?

[Posted September 1, 2004 by corbet]

Last week's USB subsystem patch posting included a number of changes, from some data structure shrinking and continuing improvements to the new USB OTG support. A less welcome part of that patch, however, was the removal of the Philips webcam (PWC) driver, by the request of its author.

The PWC driver had a unique feature in the form of a hook which allowed the loading of a binary module into the driver. This hook was used to load a closed-source decompression routine, needed to use the camera in its higher-resolution mode. This sort of hook goes against kernel policy, so, when USB maintainer Greg Kroah-Hartman became aware of it, he prepared a patch to take it out. The PWC driver maintainer (Nemosoft Unv.), in response to this change, requested that the driver be removed altogether. He has also removed the driver and all the related files (including the binary-only part) from his web site.

Greg Kroah-Hartman's FAQ on the removal of the PWC driver is worth reading.

The reaction in parts of the community has been quite strong. This is, according to some, another example where licensing fundamentalists have, through their intolerance of binary-only modules, cost Linux users the ability to work with their cameras. The PWC driver, which was not hurting anyone, has been needlessly lost. Linux will never be able to compete with Microsoft or be taken seriously by vendors as long as this kind of silliness is going on. And so on.

Whether Linux developers should be concerned with "competing with Microsoft" is a topic for a different article. For now, let us look at the issue of proprietary modules, and the kernel developers' approach to them. The general attitude toward proprietary modules is overtly hostile. Critics claim that this attitude is the result of blind ideology which puts free software fundamentalism above the needs of Linux users. The truth of the matter is that there is no end of solid, practical reasons for discouraging the creation and use of binary-only kernel modules.

The first of these is that the copyright status of many of these modules is ambiguous at best. Any module which is a derived product of the kernel must carry a GPL-compatible license; no exception to the GPL for loadable modules exists. A serious legal challenge to the distribution of a proprietary module has not yet been made. Yet. There may yet come a day, however, when one of the many holders of copyrights on kernel code decides that a binary module violates his or her copyrights, and does something about it.

Binary modules are, by their nature, platform-specific. One of the strengths of Linux is the freedom of choice it gives with regard to hardware, but binary modules take that freedom away. Linus Torvalds put it this way:

The fact is, Linux has been a hell of a lot more successful at moving to things like x86-64 and ppc64 than Windows will _ever_ be. And the reason is open source drivers.

Non-free drivers lock users into specific architectures.

When binary modules have bugs, there is no way to even track them down, much less fix them. A bad module brings down the entire kernel with it, making Linux appear to be unstable. And closed-source modules tend to have a much higher rate of bugs than free modules; they have been seen by very few eyes, rarely conform to kernel programming conventions, and their authors cannot be educated on how to do things right. A system which contains proprietary modules is less stable, and there is nothing that the kernel developers can do about it.

Closed-source modules break when the system is upgraded. The internal kernel interfaces can be changed at any time, a longstanding policy which exists for several good reasons. In-tree modules are fixed quickly; proprietary modules are fixed when the vendor gets around to it, if ever. A binary module has no future beyond whatever promises the vendor may have made regarding its support plans. Some of the more cynical among us have been known to mutter that such promises have, on occasion, not been kept. And those promises tend to be minimal in the first place; technology manufacturers are much more interested in getting people to buy new hardware than supporting their old, obsolete products.

Perhaps more to the point: binary modules are a drag on the development of the kernel. Whenever a kernel change breaks those modules, users complain loudly. The kernel developers express their lack of worry about breaking binary modules in a very clear way, but the fact is that they (and their employers) have to think before making that sort of incompatible change. Consider, for example, the change to 4KB stacks on the x86 architecture. This change makes the kernel more stable in a number of ways. But it broke the binary nVidia modules, leading to a loud chorus of protests. To the extent that those users' complaints are heard, important kernel improvements will be delayed or blocked.

Binary-only modules lack transparency; users never really know what is going on inside. There is speculation that the PWC decompression code is closed-source because opening it would reveal that the camera has far less resolution than advertised. This is almost certainly untrue, but there is no way to look at what is going on and know for sure. The lack of transparency also makes it impossible for programmers to benefit from the work that was done on the proprietary module; there may well be useful ideas there which could be applied elsewhere in the kernel, but there is no way to know. The creator of a binary-only module is benefiting from the free software development process, but is not giving back to it.

At the 2004 Kernel Summit customer panel, the technical manager from Goldman Sachs - not a person who is likely to be inclined toward ideological licensing fancies - was in the interesting position of telling the kernel developers about the advantages of having device drivers in the mainline kernel. He pointed out that drivers which have been freed and merged into the kernel do not have the sorts of stability issues experienced by users of proprietary drivers. Even the most focused and hard-nosed of users are beginning to realize that wedging proprietary code into the kernel is not in their best interest.

It is thus in the interest of all users to discourage proprietary modules. It is not a question of irrational allergies to end-user license agreements or free software fundamentalism; it is, instead, a matter of creating the most stable and capable kernel possible. Had the kernel been a friendlier environment for proprietary code, the kernel we all use now would be less capable, less stable, and less portable than it is. When you see a proprietary module break, or (as in the case of the PWC driver) be withdrawn, what you are seeing is the risk which is inherent in the use of non-free modules, not irrational behavior on the part of the kernel developers.

to post comments

Pointless ideology?

Posted Sep 2, 2004 1:14 UTC (Thu) by james (guest, #1325) [Link]

Thank you for this.

You've expounded a number of points about which I have been thinking, and done so (as usual) with considerable clarity and style.

James.

Free software survives the original copyright holder

Posted Sep 2, 2004 4:20 UTC (Thu) by bignose (subscriber, #40) [Link] (18 responses)

In the case of this driver, the advantages of free software over non-free is made particularly clear.

The Philips Web Cam driver was in two parts: a free software driver allowing the camera to be used, and a non-free portion that gives access to extra features.

Removing the non-free portion of the PWC driver has the effect that it is lost to all its users. That will never come back, until the copyright holder changes that decision. The fact that it was non-free is what makes this so painful; users are forever at the whim of the copyright holder.

Removing the free portion of the PWC driver from the mainline kernel is far less damaging. It was released under the GPL; the original copyright holder cannot take back what was already released under that license. It was removed at the request of the author (and not via copyright enforcement). Anyone is free to take the code and add it back in. And modify it, and improve it, and share the improvements, and merge them back into the kernel, and so forth.

Thus, even for this one piece of hardware, the advantage of having a driver released as free software is made abundantly clear.

Free software survives the original copyright holder

Posted Sep 2, 2004 5:08 UTC (Thu) by gdt (subscriber, #6284) [Link] (17 responses)

It was removed at the request of the author (and not via copyright enforcement). Anyone is free to take the code and add it back in.

And what of the moral aspects of ignoring the author's explicit wish? Free software has moral objectives, so moral considerations are important.

To me the most disappointing aspect has been the shoddy treatment of the author. It hardly encourages participation in the Linux kernel community.

As an aside, if you are upset at losing support for a Philips web cam you should really be upset at Philips. They could fix this situation with a simple letter declaring public the information releasing to Nemosoft under their NDA. The whole purpose of the video compression -- to fit a webcam video stream into USB v1 bandwidth -- has gone with USBv2. Keeping the details secret is no longer going to give Philips an advantage over other hardware manufacturers.

<mischevious grin>

It will be interesting to see if the reasoning behind removing the PWC compression API is applied to various firmware download APIs in drivers. Most of those APIs only exist to facilitate the use of non-free firmware.

</mischevious grin>

Binary-only firmware

Posted Sep 2, 2004 7:29 UTC (Thu) by vmlinuz (guest, #24) [Link]

It will be interesting to see if the reasoning behind removing the PWC compression API is applied to various firmware download APIs in drivers. Most of those APIs only exist to facilitate the use of non-free firmware.

As I understand it - and I could be wrong - there has been a line drawn between firmware like this, where the kernel driver is responsible for loading the code into an external device, and a binary driver, where the closed-source code is loaded into the kernel and executed within the kernel's address space on the system CPU. In a perfect world, of course we'd like binary-only firmware to go away too, but in the same way that most people are willing to accept, say, a closed-source BIOS, where they may not accept a closed-source kernel, binary-only firmware is a much lesser evil.

Plus, of course, for those devices which *don't* have the firmware loaded by the driver, they would tend to have it in a ROM on the device anyway, so unless you're going to demand that every piece of code, microcode, firmware, etc. exectuting within your system is open, it doesn't make all that much difference if the firmware is loaded by the kernel driver, or just there permanently...

Free software survives the original copyright holder

Posted Sep 2, 2004 8:25 UTC (Thu) by hingo (guest, #14792) [Link]

And what of the moral aspects of ignoring the author's explicit wish?

What about it? The guy knowingly releases code under the GPL. He knowingly goes against the hottest debated kernel policy by secretly smuggling in hooks to be used by a binary-only module. And now we should abandon some GPL code, because he asks for it. If you ask me, Linus and Greg have been bending over backwards for a guy that was only out to screw them.

In any case, I've understood that one good reason to pull the driver, is that it's not wise to have code hanging around that doesn't have a maintainer. So when somebody picks up this piece of abandoned GPL code and offers it back to the kernel, it will be included again. At least that what I read on kerneltrap.

Ignoring author's explicit wish

Posted Sep 2, 2004 8:30 UTC (Thu) by kamil (guest, #3802) [Link] (3 responses)

And what of the moral aspects of ignoring the author's explicit wish? Free software has moral objectives, so moral considerations are important.

Fair enough, but what about the users who paid for the camera, and want to use it?

It might not be the nicest thing to do to ignore the author's wish, but ignoring the clear interests of the users sounds a lot worse to me.

If you release something under GPL, you lose complete control over the code. If you don't like it, don't release under GPL in the first place.

Ignoring author's explicit wish

Posted Sep 2, 2004 13:30 UTC (Thu) by smoogen (subscriber, #97) [Link] (2 responses)

>Fair enough, but what about the users who paid for the camera, and want to
>use it?

They are also free to pay for someone to support the product for them or use another OS. There is a 'price' to using Linux.. if you dont like that price find an OS that meets your bill better.

People seem to have missed the various parts where Greg said, you are free to take the code.. fix the broken bits and resubmit it.. just be prepared to maintain it. [or they did see it and just dont want to maintain it.]

Ignoring author's explicit wish

Posted Sep 2, 2004 15:08 UTC (Thu) by kamil (guest, #3802) [Link] (1 responses)

I basically agree with what you're saying.

The only problem is, that what you're saying is off-topic for this particular thread. Go read the posting #100434 I was replying to. I basically disagreed with the notion that if an author of a GPL software "withdraws" it, we should all obey. It might be a polite thing to do, but polite is not necessarily the same as right.

Ignoring author's explicit wish

Posted Sep 2, 2004 20:05 UTC (Thu) by iabervon (subscriber, #722) [Link]

He was also the maintainer of a device driver, and he wanted a change to it applied. As his change doesn't cause problems for any other code, it gets applied, obviously. If someone else wants to take over maintainership, they can, and, since the old driver was under the GPL, they can base their driver on it. They should probably name it something different, though, because it won't behave as users expect (i.e., pwcx won't load). In fact, they should arrange it such that decompression can be done in userspace like is generally considered correct.

Free software survives the original copyright holder

Posted Sep 2, 2004 11:56 UTC (Thu) by Fats (guest, #14882) [Link] (1 responses)

And what of the moral aspects of ignoring the author's explicit wish? Free software has moral objectives, so moral considerations are important.
To me the most disappointing aspect has been the shoddy treatment of the author. It hardly encourages participation in the Linux kernel community.

The author should not have released it's software as GPL in the first place then. He should have thought about it before commiting. When you are commiting code to the linux kernel you have to be aware that your code becomes part of the kernel and some patches may be accepted that you don't like. If you can't accept that don't code in GPL.

Free software survives the original copyright holder

Posted Sep 2, 2004 13:01 UTC (Thu) by fjf33 (guest, #5768) [Link]

I think the author was more worried about having to deal with all the email and support coming from people that cannot make the camera work. Because without the compression the camera is very crippled.

In hindsight, if he was not trying to make a statement, he should have supplied a patch that removes it from the kernel and be quiet about it. Put a note that he is not maintaining it anymore and leave it at that.

Free software survives the original copyright holder

Posted Sep 2, 2004 16:11 UTC (Thu) by AJWM (guest, #15888) [Link]

And what of the moral aspects of ignoring the author's explicit wish?

Just which of the author's explicit wishes are you referring to? This latest one, or the wish the author expressed -- repeatedly -- when he first released (part of) the driver code, and subsequent modifications, under the GPL?

To me the most disappointing aspect has been the shoddy treatment of the author.

To me the most disappointing aspect has been the author's shoddy treatment of the community. From the second sentence of the preamble to the GPL: "the GNU General Public License is intended to guarantee your freedom to share and change free software" [emphasis added]. What kind of a guarantee is it if we let the guarantor say "sorry, I've changed my mind, you can't share this any more"?

If he no longer wants his name associated with it, that's certainly his perogative (especially if he no longer wants to be annoyed by support requests or bug reports). But not saying, in effect, "I'm taking my ball and going home, you're all big meanies and I don't want to play anymore".

I think we ought to abide by the wish the author expressed when he licensed the code under the GPL in the first place.

Free software survives the original copyright holder

Posted Sep 2, 2004 16:20 UTC (Thu) by iabervon (subscriber, #722) [Link]

It will be interesting to see if the reasoning behind removing the PWC compression API is applied to various firmware download APIs in drivers.

That question has already come up, and the answer was that firmware is software for a different computer (in fact, there are hard drives with chips which are CPUs on other computers), and is therefore installing it through the kernel is not really different from sending a ISO image in response to an HTTP request with sendfile(); the only difference is that the other computer is connected by PCI and it's not far away.

The issues with binary drivers don't apply, as well. Changing the kernel behavior doesn't matter to the firmware; changing the host architecture doesn't matter either. The firmware might actually be written in something not usefully processable by end users (e.g., FPGA images aren't going to be written in anything that gcc handles).

Debian removed binary blob drivers

Posted Sep 2, 2004 16:54 UTC (Thu) by pflugstad (subscriber, #224) [Link] (5 responses)

It will be interesting to see if the reasoning behind removing the PWC compression API is applied to various firmware download APIs in drivers. Most of those APIs only exist to facilitate the use of non-free firmware.

Debian's already done this - their latest distro kernels (as of about 2.6.4 or so) have removed ALL support for drivers that contain binary blobs in them.

I'm of two minds about this: one, I agree with the idea; two, it's a major pain in the ass from a simple users point of view.

Debian removed binary blob drivers

Posted Sep 2, 2004 17:14 UTC (Thu) by bronson (guest, #4806) [Link] (4 responses)

Does this make any sense at all?? Do these debian guys also disable the boot ROMs on their network and video cards? Do they unplug the BIOS chip from their motherboards? There is absolutely no difference between a closed-source firmware blob loaded over an I2C bus (say, a serial EEPROM) and a closed-source firmware blob loaded over the PCI bus. None at all.

I'm not happy to see this since I use Debian the most. Can anybody explain their thinking?

Why Debian removed binary blob drivers

Posted Sep 3, 2004 0:53 UTC (Fri) by abredon (guest, #2038) [Link] (2 responses)

Yes, this makes sense under the Debian social contract - the main archive can only contain software under DFSG-free licenses, period. Binary uploadable firmware is proprietary software, and therefore cannot be included in the main debian archive - it has to go in the non-free archive.

The only options available under the Debial social contract are:
1. remove proprietary firmware from the kernel archive, or
2. move the entire linux kernel to non-free.

The second choice is unpleasant, as that means that there is no way to build a Debian system with only Free Software. Therefore Debian has opted to remove the non-free firmware from the kernel archive. (it may still be available in non-free)

Debian does not distribute hardware, so (for example) the ROM in a hard drive is not affected by the social contract.

Why Debian removed binary blob drivers

Posted Sep 3, 2004 15:05 UTC (Fri) by bronson (guest, #4806) [Link] (1 responses)

That's the argument?? "It's in the social contract, my hands are tied." Blind gestapo adherance to Debian guidelines? Did anybody think of how this would affect the user? Or free software in general?

I loathe hardware compatibility issues, especially pointless ones. Looks like it's time for me to start looking at different distros...

Why Debian removed binary blob drivers

Posted Sep 5, 2004 7:31 UTC (Sun) by piman (guest, #8957) [Link]

It's not "blind adherence." Repeated votings have shown that Debian developers want binary blobs out of the distribution, and/or consider them a violation of the Social Contract. No DD has yet proposed a change to the SC to allow them, either.

Debian removed binary blob drivers

Posted Sep 3, 2004 2:00 UTC (Fri) by bignose (subscriber, #40) [Link]

> Do these debian guys also disable the boot ROMs on their network and video
> cards? Do they unplug the BIOS chip from their motherboards?

That's outside the realm of what Debian distributes. Debian doesn't distribute any non-free boot ROMs (because they don't distribute boot ROMs). Debian doesn't distribute non-free BIOS (because they don't distribute any BIOS).

If free boot ROMs, or BIOS code, becomes available and useful to Debian users, Debian might start distributing those too.

> There is absolutely no [software-freedom] difference between a
> closed-source firmware blob loaded over an I2C bus (say, a serial EEPROM)
> and a closed-source firmware blob loaded over the PCI bus. None at all.

That's right. Which is why Debian doesn't distribute any of them.

Free software survives the original copyright holder

Posted Sep 16, 2004 22:30 UTC (Thu) by job (guest, #670) [Link]

most disappointing aspect has been the shoddy treatment of the author

This is just too stupid. What do you think about all the people who have sent patches, bugreports, or just helped with answering his questions to get the driver in shape? They never assigned any copyrights. And what about the skeleton drivers he probably used to get started? He doesn't own the copyrights to that either.

No, you can't simply take your toys and leave. The Linux copyright assignment explicitly isn't one, so all authors owns a piece of the big cake, making that impossible.

Pointless ideology?

Posted Sep 2, 2004 4:32 UTC (Thu) by einstein (subscriber, #2052) [Link] (9 responses)

I think this is a concise and, for the most part, realistic discussion of
the issues. There is one glaring exception to the general trend of
reliability in open source vs proprietary drivers though - the canonical
exception to the rule if you will:

Nvidia - their video drivers are absolutely the best available for linux -

I have had severe problems with crashes/lockups using ATI cards and the
open source DRI drivers, to the point that Linux + ATI was an unusable
combination for accelerated 3D - I've seen this severe ATI stability
problem beginning in 2001 with 3D accelerated FPS games on redhat 7.x, on
up to a few months ago, when fedora core 1 would lock up if you dared to
run an opengl screensaver with an ATI card. In all cases, the same systems
that were locking up with the ATI cards have been rock solid with Nvidia
cards and their opengl drivers.

As far as performance, Nvidia is the best bang for the buck by far for
linux. One indication is that with Nvidia cards, the linux opengl
performance tends to be every bit as good as on ms windows.

As fas as being linux-savvy, Nvidia, while not perfect, is miles ahead of
any other vendor. Nvidia are very responsive to the linux community, and
their drivers are well maintained and kept up to date, to the minute. When
installing suse 9.1, I had only to check the box that says "install nvidia
gl drivers" and it was done - no editing of config files, just working
accelerated 3D next time X was started.

So when we discuss these issues, let's give credit where credit is due, as
Nvidia works damn hard to be the vendor of choice - and they've earned my
business.


Pointless ideology?

Posted Sep 2, 2004 4:47 UTC (Thu) by BrucePerens (guest, #2510) [Link] (1 responses)

When last I checked, the closed nvidia driver for AMD64 did not provide 32-bit emulation, as the entire rest of the kernel interface did. Previously, a trivial problem with setting the card's gamma remained unfixed for years, seriously inconveniencing many filmmakers.

The folks who really depend on these drivers, the visual effects industry, have to walk on eggs around nVidia and ask very politely for support, because they can't get it elsewhere and their market isn't so important to nVidia.

We really need to fix that.

Bruce

Pointless ideology?

Posted Sep 9, 2004 12:48 UTC (Thu) by forthy (guest, #1525) [Link]

The current nVidia driver for AMD64 does support 32 bit emulation. It
wasn't in there right from start, but they figured it out.

For a closed source driver, nVidia is almost as good as it can get. The
point that an open sourced nVidia driver would be much better is
undisputed.

Pointless ideology?

Posted Sep 2, 2004 5:43 UTC (Thu) by evgeny (subscriber, #774) [Link]

I had exactly the opposite experience with NVidia and their support, which was as responsive as /dev/null after a kernel crash (which happened twice a day). After a month of pointless struggle, I swapped the card for an ATI one, supported by X.org out of the box and has been happy since then. Your luck with NVidia is explained by a relative popularity of Linux/i386. My box is amd64 and they obviously didn't care much about it.

Please note that this post is not about which vendor is better (ATI hasn't released a 64bit driver at all); it's about the inherent devil of non-free drivers and software in general.

Pointless ideology?

Posted Sep 2, 2004 8:41 UTC (Thu) by ken (subscriber, #625) [Link] (1 responses)

Nvidia is a very good example of the problem with linux drivers.The driver only works on x86 hardware no driver for powerpc exist. Had the driver been open it would work on any platform.

Pointless ideology?

Posted Sep 2, 2004 12:45 UTC (Thu) by wookey (guest, #5501) [Link]

Yep - no drivers for ARM either, which severely limits the use of these cards on that platform. I am hopeful that the increasing diversity of architectures in popular use will reduce the incidence of binary modules, but it looks like it'll be a while yet in video-card land which seems particularly badly afflicted for some reason.

Pointless ideology?

Posted Sep 2, 2004 16:24 UTC (Thu) by AJWM (guest, #15888) [Link] (3 responses)

Nvidia - their video drivers are absolutely the best available for linux

No they're not, even if you confine your comparison to drivers for nVidia cards (vs the universe of Linux drivers or even video drivers).

They don't work worth a damn, for example, on Linux-PPC. And as Bruce points out above, their AMD64 drivers don't work well either. And lord help you (because nVidia won't) if you want to run one of their older PCI cards (Riva, TNT, etc) under Linux even on an x86 box, let alone PPC or SPARC or MIPS.

No, the nVidia drivers are a prime example of the insidious nature of captive binary-only drivers. You think you've gotten a great deal. Just wait until nVidia decides your card is no longer worth their time to support, or you want to move it to a different architecture machine.

Pointless ideology?

Posted Sep 2, 2004 18:34 UTC (Thu) by einstein (subscriber, #2052) [Link] (2 responses)

They don't work worth a damn, for example, on Linux-PPC. And as Bruce points out above, their AMD64 drivers don't work well either. And lord help you (because nVidia won't) if you want to run one of their older PCI cards (Riva, TNT, etc) under Linux even on an x86 box, let alone PPC or SPARC or MIPS.

You and others have a point about non-x86 platforms. For ppc et al, nvidia is not a good choice now - but that could change, especially if "friends" like IBM were to lobby on our behalf for better ppc linux driver support.

However you are dead wrong about nvidia support for older cards - one nvidia driver supports all nvidia cards - from the lowly vanta to the newest high end card.

Pointless ideology?

Posted Sep 2, 2004 21:52 UTC (Thu) by AJWM (guest, #15888) [Link] (1 responses)

For ppc et al, nvidia is not a good choice now

Yet several models of PowerMac ship with nVidia cards (the high end ships ATI). Guess I'm out of luck if I want to try Linux on them.

one nvidia driver supports all nvidia cards [on x86]

You may be right about Quadro, GeForce and TNT2 cards -- although I can't comment on the quality of the drivers.

But for Riva and TNT (not TNT2) cards, see http://www.nvidia.com/object/riva_drivers and note that only old versions of Windows, BeOS, and OS/2 are supported. (The OS/2 driver also supports Vanta and TNT2.)

Pointless ideology?

Posted Sep 2, 2004 22:11 UTC (Thu) by einstein (subscriber, #2052) [Link]

But for Riva and TNT (not TNT2) cards, see http://www.nvidia.com/object/riva_drivers and note that only old versions of Windows, BeOS, and OS/2 are supported. (The OS/2 driver also supports Vanta and TNT2.)

Ah, so the most ancient nvidia cards are not covered by their unified driver? my mistake, I thought that was the case, but apparently the tnt 2 are the oldest supported cards. Then again, don't the riva and tnt cards work in 2D?

If I were looking for 3D support on linux, I wouldn't start with an ancient riva or tnt - tnt 2s are available online for $16, or if you want a good older 3D card with open source drivers, voodoo 3s are still available quite cheaply if you know where to look.

IBM

Posted Sep 2, 2004 4:48 UTC (Thu) by BrucePerens (guest, #2510) [Link]

At the Linux Journal advisory board dinner, Dan Frye of IBM said that an important job for the magazine was to convince people to stop providing binary device drivers. Apparently they are a support headache for mighty IBM, too.

Bruce

OT nitpick [resolution]

Posted Sep 2, 2004 5:46 UTC (Thu) by guybar (guest, #798) [Link] (4 responses)

[supposedly] ... the camera has far less resolution than advertised. [...] there is no way to look at what is going on and know for sure.

Independently of the rest of the fine article, this is trivially not true: experimentalists commonly measure camera resolutions, w/o looking at any source code.

I'm sure most of LWN's readers, having common-sense and a printer, can think-up elementary ways of doing so, at webcam resolutions.

OT nitpick [resolution]

Posted Sep 2, 2004 16:32 UTC (Thu) by AJWM (guest, #15888) [Link] (3 responses)

On the contrary, if the camera firmware were to do pixel interpolation, it could fake a higher resolution than it actually has. True, a sufficiently intelligent image analysis program might be able to detect this and deduce the real resolution, but I can think of a couple of easy ways to make that difficult. (Eg add a bit of randomness to the interpolated pixels. Add a sharpness transform to the decompression code in the driver. That kind of thing.)

Other than examining the complete driver source code (and maybe not even then), the only real way to be sure is to reverse engineer the hardware, and that's only easy (relatively speaking) if the manufacturer uses standard parts and doesn't conceal the part numbers.

OT nitpick [resolution]

Posted Sep 3, 2004 12:43 UTC (Fri) by kreutzm (guest, #4700) [Link] (2 responses)

Well, what really matters is the resolution the application receives,
isn't it? This is measured routinely for e.g. still cameras using test
pictures. One of those has, e.g., parallel lines with decreasing distance.

So you should be able to ask a photographer near you (or the lab of your favourite computer magazine) to check for it. Either by providing them the camera or borrowing the test picture and providing them the raw (ie. uncompressed) image.

OT nitpick [resolution]

Posted Sep 3, 2004 17:12 UTC (Fri) by ekj (guest, #1524) [Link] (1 responses)

You are correct, offcourse, but the thing is, that's not the sort of "resolution" marketed by camera-manufacturers.

My digital Camera is marketed as delivering pictures with a "resolution" of 2500x1800 or something of the sorts, that means simply the numbers of pixels coming out of the camera. There's no way in hell you'd be able to take a picture of a black-white checkerboard with that number of squares on it, and have the individual squares resolve black/white, that picture would appear a fairly uniform grey. (and the fact that the camera outputs lossy jpeg ain't the only reason for that)

My limited, admittedly amateur testing seems to show that the real "resolution" in the sense of "can resolve an object this small" is at most half of the advertised resolution.

OT nitpick [resolution]

Posted Sep 16, 2004 22:24 UTC (Thu) by job (guest, #670) [Link]

The pictures that digital cameras take are interpolated. A camera that
advertises 6 Mpixels is 2 M red senors, 2 M green and 2 M blue. (Strictly
not true of course, there are more blue sensors than red I think.) So the
resolution is indeed correct but the colors are all interpolated.
Measuring this in the way you describe is probably pretty difficult.

This is true for all cameras, consumer all the way up to pro. Resolution
may be further damaged by crappy optics, but this is of course less true
the more high end you get. There is one sensors that actually uses a true
3 Mpixels resolution, from a company called Foveon, but as far as I know
only Sigma actually made a camera with that. It turned out to be a hard
sell because of the perceived lower resolution (you know how it is, the
3GHz CPU must be faster than the 2GHz one), so in the end they renamed it
to 9 Mpixels just to go with the lingo.

Yet another binary driver problem

Posted Sep 2, 2004 7:52 UTC (Thu) by mbp (subscriber, #2737) [Link]

I was just struggling with this today, before giving up and going to read LWN.

One of my machines has an ATI FireGL2 card in it. A couple of years ago this was apparently quite an expensive and powerful card. (Not that I really care; I only run mozilla, rxvt and emacs, but it came in the machine.)

However, ATI are apparently now bored with supporting it: the last driver release was in November 2002 for XFree86 4.2. I don't want to keep running the same kernel and X software I had in 2002, so as time goes on it's gradually getting worse. This card doesn't seem to support VESA modes. It's now hobbling along in a very slow 2d mode. I suppose someday I need to replace it with a cheap Radeon that has at least open-source 2d support.

ATI's answer seems to be that people ought to buy an expensive new card if they want an updated driver, otherwise they can see figure 1.

Damned if I'll buy anything that needs proprietary drivers in the future if I can possibly avoid it.

One quibble

Posted Sep 2, 2004 8:00 UTC (Thu) by mbp (subscriber, #2737) [Link]

To be fair, this problem of the driver being pulled out is not *entirely* due to licencing.

Two things happened here:

1. Greg K-H noticed the hook to support non-free code, and removed it.

2. In response to that, Nemosoft asked that the entire driver be pulled out.

Although the provocation was a conflict over closed code, Nemosoft's actions are entirely independent of the licence. The uncompressed-data driver was removed because the author wanted it removed, and Linus and Greg decided to respect that. Nemosoft couldn't force it to come out, but it seems the maintainers thought it was good to do what he wanted. I think I'd do the same; it's bad form to go against the wishes of the author even if it's technically legal.

That could happen with anything else: if a kernel flamewar pushes somebody over the edge we might lose a filesystem one day, for example. Or at least, the author might disown it. I think if it's a significant feature then someone will eventually pick it up and fork it, which may happen with this driver.

Reverse engineering?

Posted Sep 2, 2004 9:46 UTC (Thu) by rwmj (guest, #5474) [Link] (1 responses)

How big is this "compression routine"? Can it not be reverse engineered?

Rich.

Reverse engineering?

Posted Sep 2, 2004 11:14 UTC (Thu) by dvrabel (subscriber, #9500) [Link]

Had the driver not been available as a binary (and thus usable by 98% of people) then I suspect that the compression format would have been reverse engineered already.

As an example, Zoran's USBVision chip (ZR36504/ZR36505, formerly NT1003/NT1004), used in some USB TV capture devices, had its compression algorithm reverse engineered by someone reading academic papers on video compression algorithms.

Pointless ideology?

Posted Sep 2, 2004 10:09 UTC (Thu) by ballombe (subscriber, #9523) [Link] (2 responses)

Looking at the fact in the PWC story, the morale is not about binary modules,
it is "Never sign a NDA". I am sure Nemosoft would be more than happy to
opensource the binary driver, but the NDA prevent it.
So if you contemplate writing a device driver for Linux the first rule is, do
not sign any NDA, that can only make things _more_difficult.

Pointless ideology?

Posted Sep 2, 2004 11:26 UTC (Thu) by dvrabel (subscriber, #9500) [Link] (1 responses)

The NDA has expired[1] and I remain unconvinced by Nemosoft's arguments regarding not opening the source -- seeing as in the past 2 months he appears to have not contacted Phillips to clarify.

[1] http://www.smcc.demon.nl/webcam/

Sneakiness and Arrogance

Posted Sep 2, 2004 16:19 UTC (Thu) by ncm (guest, #165) [Link]

I agree. Nemosoft objects to the kernel policy. He sneaked in a violation, and then tried to force Linus to continue allowing it. Linus refused to be blackmailed, and was only too happy to get those dirty fingerprints out of his kernel.

Evidently, this guy doesn't care about anybody who has the camera; his own ideological power games are more important to him. The clincher is the NDA: if it weren't about power games, he could have just GPLed the decompression code too.

It's just too bad that this didn't happen three years ago, so that people wouldn't have bought those cameras thinking they were properly supported.

[OT] What web camera *is* well supported by the stock kernel?

Posted Sep 2, 2004 10:19 UTC (Thu) by nicku (subscriber, #777) [Link] (1 responses)

Thanks for the article.

I gave up and bought an expensive composite out video camera and btv capture card. Has any one found a cheap USB webcam that *is* well supported by the stock kernel? All USB cameras I've tried on Linux had painfully slow frame rates.

[OT] What web camera *is* well supported by the stock kernel?

Posted Sep 2, 2004 16:08 UTC (Thu) by stuart2048 (guest, #6241) [Link]

Another option is an Axis network cam. Bonus: they run Linux.

Pointless ideology?

Posted Sep 2, 2004 11:08 UTC (Thu) by bchilds (guest, #23751) [Link] (2 responses)

I'm on board for any way to kill binary only modules.

Doesn't microsoft claim that a lot of their past stability problems were due to poorly engineered third party driver (aka binary only drivers).

However, I remain confused about the position Linus is taking with respect to these modules. He has the power to declare that they are not allowed because they violate the GPL.

What I've heard is that he's somewhat tolerant of them though.

Can anyone shed some light on this issue?

Pointless ideology?

Posted Sep 2, 2004 11:42 UTC (Thu) by flewellyn (subscriber, #5047) [Link]

As far as I know, it's an issue of whether the binary driver is a derivative work of the kernel, or not. If it's a body of code that was mostly written on another platform, and only the interfaces were ported to the Linux kernel, then no, it probably isn't derivative. (See the nVidia driver, for instance.) On the other hand, if the module in question uses a lot of internal kernel interfaces, hooks into various other subsystems, etc, then it probably IS derivative. This is a very tricky issue, though.

Pointless ideology?

Posted Sep 2, 2004 12:10 UTC (Thu) by Fats (guest, #14882) [Link]

However, I remain confused about the position Linus is taking with respect to these modules. He has the power to declare that they are not allowed because they violate the GPL.

Unfortunately it is not. The only authority to decide if something is against a license like the GPL is a judge. Linus can only tell how he sees that the kernel license has to be interpreted. Then he has to be careful that his comment is not treated as legal advice because from what I heard this can have consequences in the USA.

Also he can't put more restrictions on the license then it is now without the approval of all kernel contributors which is I think an undoable task; certainly not worth the effort.

API/ABI stability

Posted Sep 2, 2004 13:46 UTC (Thu) by elanthis (guest, #6227) [Link] (1 responses)

ABI/API breakage hurts *all* users, even ones with hardware supported by Free drivers. Any new hardware driver is only going to be in a newer kernel. Users running a distribution even a couple months old may find that there is some new hardware they wish to use, the driver can't be used with their kernel due to API breakage, and they are forced to upgrade their kernel. Which means they have to take any other new behaviours, breakages, user-space tool modifications, and so on just to get some new periphial working. With ABI breakage, users are always forced to recompile drivers. If you upgrade a kernel, you have to remember to recompile and reinstall all drivers, possibly regenerate your initrd images, etc. *IF* there was a standard kernel source installation mechanism that found out-of-tree drivers and rebuilt them and regenerated the initrd, such that any kernel driver would be handled by any distro kernel packages or kernel rebuilds, this wouldn't be a problem.

API/ABI stability

Posted Sep 2, 2004 16:38 UTC (Thu) by piman (guest, #8957) [Link]

> *IF* there was a standard kernel source installation mechanism that found out-of-tree drivers and rebuilt them and regenerated the initrd, such that any kernel driver would be handled by any distro kernel packages or kernel rebuilds, this wouldn't be a problem.

I don't know about "standard", but Debian has such a program. It builds .debs for any kernel source tree and additionally .debs for any module source packages.

Pointless ideology?- ndiswrapper

Posted Sep 2, 2004 14:24 UTC (Thu) by jva (guest, #7172) [Link] (4 responses)

As much as I'd prefer to use hardware with open-source drivers, it's sometimes hard to avoid binary-only drivers. When I investigated 802.11 b/g wireless cards about 2 months ago, I couldn't seem to find more than 1 or 2 802.11 b/g cards supported by Linux. (Perhaps I didn't look in enough places.)

By using ndiswrapper (http://ndiswrapper.sf.net), I could buy a $50 wireless card from Dell, and use ndiswrapper to load the provided Windows driver. This was a practical solution that greatly increased the number of choices I had for wireless networking while still using Linux.

Pointless ideology?- ndiswrapper

Posted Sep 2, 2004 17:11 UTC (Thu) by AJWM (guest, #15888) [Link]

When I investigated 802.11 b/g wireless cards about 2 months ago, I couldn't seem to find more than 1 or 2 802.11 b/g cards supported by Linux.

How many do you need?

Pointless ideology?- prism54 for 802.11g

Posted Sep 2, 2004 17:43 UTC (Thu) by kimoto (subscriber, #5244) [Link] (2 responses)

Many 802.11g cards are (allegedly) supported by the prism54 driver, in the stock kernel since 2.4.28-pre1. You have to tolerate a binary firmware blob, though. The web list of supported hardware seems to be unavailable right now, but perhaps you can search for the right chipset(s) at linux-wlan-ng. (I use the Netgear WG511 cardbus adapter.)

Pointless ideology?- prism54 for 802.11g

Posted Sep 2, 2004 17:45 UTC (Thu) by kimoto (subscriber, #5244) [Link]

... and in 2.6 since 2.6.5.

Pointless ideology?- prism54 for 802.11g

Posted Sep 9, 2004 10:40 UTC (Thu) by cortana (subscriber, #24596) [Link]

Just a word to anyone thinking of buying a card supported by prism54: if the card you are looking at has the SMC 2802W chipset, make sure it is not the 'v2' revision; the 2802Wv2 is not compatible with the prism54 driver. I only found this out after buying the card. Fortunatly, I was able to use the Windows drivers with Ndiswrapper.

Pointless ideology?

Posted Sep 2, 2004 16:05 UTC (Thu) by allesfresser (guest, #216) [Link] (4 responses)

I think it's a good thing that many people are coming to realize that "free software fundamentalism" actually does have a good reason for existing. It wasn't revealed on golden tablets or anything... it comes from hard experience with proprietary vendors who care only about selling their products and not about supporting their users. (Fill in the name of the proprietary vendor that comes to your own mind...)

Freedom should be the primary and overriding concern when it comes to any decision about including or excluding code... 'market share' and 'competition with Microsoft' take a fourth-row back seat as far as I'm concerned. This episode only proves it very nicely.

Pointless ideology?

Posted Sep 2, 2004 21:57 UTC (Thu) by AJWM (guest, #15888) [Link]

I seem to recall a story about the whole thing getting started when some guy wanted to modify a printer driver or some such and the vendor wouldn't give him the specs. Richard Stallman, I think his name was.

Pointless ideology?

Posted Sep 3, 2004 2:08 UTC (Fri) by paulusbd (subscriber, #6253) [Link] (2 responses)

I also agree to that having this discussion is very good. However, I think there is a consideration that has not been presented.

I still don't understand why there is this feeling that software must absolutely be free. There is always an exception to everything and I have never heard of a software or hardware developer who has fed themselves or their family by giving away software that took their creativity and hard work to develop. I, however, very much agree that the software community should have the freedom to choose which software they use, free, not free, or both; all while keeping within a framework that is fair to those who use it.

A point in this discussion that I have not noticed, perhaps I'm just in need of new glasses, is the one that deals with proprietary hardware. Most Non-GPL-Compliant Modules are for hardware that has a great deal of custom design to it, if you will. The Non-GPL Modules are compiled by the Vendors (Nvidia, Promise, and others) who wish to keep the inner workings of their ASICs a secret for business reasons. It is a real shame that some type of API mechanism could not be developed to allow a wrapper type driver to keep GPL Kernel Derived code separate from an interface that should remain proprietary and allow it to do so in such a manner as to satisfy all and allow the device in question to operate. I have worked for several companies that have developed specialized hardware for Windows and UNIX that also wished to port a driver to Linux. I was fortunate enough to be involved in a few of these ports and gained some insight into the “Whys” of the issue. The problem we ran into was how to make a driver that exploited the capabilities of the hardware without broadcasting the trade secrets that made the hardware unique to all of our competitors in the form of reverse-engineerable source code. A great number of secrets can be revealed, including flaws (I challenge any of you to develop an ASIC without flaws, even after 3 to 4 rolls) in how certain code is written to 'talk' to ASICs.

If this were not such a worry, the issue with modules would not be an issue. I am just saddened by the attitude of "We don't want any proprietary stuff" when it leaves people out in the cold for no good reason (I know plenty of good arguments have been raised in the issue of closed source software, but hardware interfacing is another issue). A compromise must be struck on this issue or Linux will be deprived of a great number of useful inventions. A great number of us love Linux and what it stands for, but we don't want to give up our toys either! I think the sentiment of "Find another O/S if you want to use that" is just as exclusionary as what closed source software is accused of being.

There are very good arguments on both sides of this issue and all should be considered. What needs to be done is the creation a good solution that would satisfy the development, legal, and end-user communities while allowing companies to keep their secrets.

-Paul D.

Re: Pointless ideology?

Posted Sep 5, 2004 8:05 UTC (Sun) by gleef (guest, #1004) [Link] (1 responses)

I still don't understand why there is this feeling that software must absolutely be free.

You don't need to use software that's absolutely free if you don't want to. However, for legal and maintenance reasons software developers and distributors need to be able to deal with clean, absolutely free code. If the Linux Kernel distribution contains non-free code, they face lawsuits and huge maintenance headaches, and lose developers. If the Debian distribution contains non-free code, they lose developers and distributors. If it doesn't matter to you, don't worry about it for your stuff, but let the people who proprietary software harms worry about it for their stuff without griping about the decisions they make.

The Non-GPL Modules are compiled by the Vendors (Nvidia, Promise, and others) who wish to keep the inner workings of their ASICs a secret for business reasons. It is a real shame that some type of API mechanism could not be developed to allow a wrapper type driver to keep GPL Kernel Derived code separate from an interface that should remain proprietary and allow it to do so in such a manner as to satisfy all and allow the device in question to operate.

As far as I know such a wrapper could be written. However, the Linux maintainers have wisely and practically declined to include such a module; if this wrapper were to be in the main kernel, it would be a nightmare. First, including a wrapper would encourage binary drivers, which notoriously include poor or missing support for non-mainstream hardware; for example, NVidia drivers have poor AMD64 support and no PPC support.

Next, this wrapper would have legal implications. Binary drivers in the Linux kernel are debatably a contract violation. The wrapper would potentially make it harder to enforce the GPL.

Also, the kernel developers would increasingly be faced with the choice of making a change to the kernel and dealing with outcry from broken binary drivers, or slowing kernel development. This is a lose-lose choice that kernel developers want to stop confronting; your suggestion would have them deal with it more often.

Proprietary code might have a place in the world, however, that place isn't stashed inside Free Software.

Re: Pointless ideology?

Posted Sep 7, 2004 22:32 UTC (Tue) by paulusbd (subscriber, #6253) [Link]

Excellent reply!

This is a much more understandable reason than that of an ideology issue. Shame on me for not doing my due diligence and homework.
I suppose most problems could be worked around as long as the problem is just the one with distribution issues. Unfortunately, I do see the storm brewing about the proprietary modules being developed with GPL Kernel interface code. This is a more complex issue and one that I would hope could be addressed.

I couldn't agree with you more about the lack of support and development for a great number of these devices. The companies I have worked with always seem to get into these ruts and they completely loose interest in the Linux port. This keeps the driver stuck at whatever Kernel level it was built for when the interest left the building. Keep in mind that it is the company interest I am talking about, not the developer's. A great number of us would love to continue our work, if only for the idealist reason that it should be done and be state-of-the-art :) It is painful to hear the complaints and know the shortcomings of the code that is out there while being greatly hindered in any effort to correct the situation. Silly money and profit issues, bah! And you are right about the API issue; the Linux kernel does change often enough in its interfaces as to put a great strain on those who must maintain the API.

Quite true, Proprietary code shouldn't be hiding inside, nor stashed.

-Paul D.

Get up to date: Linux: Restoring the Philips Webcam Driver

Posted Sep 2, 2004 16:56 UTC (Thu) by southey (guest, #9466) [Link]

See Linux: Restoring the Philips Webcam Driver at http://kerneltrap.org/node/view/3747

How to ship an FPGA's code

Posted Sep 7, 2004 0:20 UTC (Tue) by gerardmcglew (guest, #3805) [Link] (2 responses)

Imagine a graphics card, with a significant piece of FPGA calculation engine. Such a card could be made to have a standard (say PCI) interface, used initially to program the FPGA, after which it behaves as a standard graphics card with hardware acceleration.

To write a driver for this card, I would have to be able to send code to it at start up - this could be an array of hex numbers, or a binary file. It would be hard to supply source code to the hardware accelerator without giving away my crown jewels, and the source code would be useless to someone without the appropriate FPGA compilation tools (ie most users).

Assuming that the rest of the driver is GPL'ed after the init code, can anyone tell me how this type of binary initialisation code can be shipped with the kernal, or with distros like Debian which don't want to ship binary chunks.

Thanks,
Gerard
ps. This graphics card is imaginary.
pps. If anyone's driver does as described, your secret is safe - I won't know until you make the information public by contacting me.

How to ship an FPGA's code

Posted Sep 7, 2004 8:19 UTC (Tue) by mwilck (subscriber, #1966) [Link] (1 responses)

There are drivers like that in the kernel that upload huge chunks of binary firmware at startup. For example, the Qlogic fiber channel driver (qla2x00).

There seems to be a consensus (well, more-or-less) that this firmware, from the kernel point of view, is simply a piece of opaque data.

However these drivers apparently don't qualify as "free" in the sense of the Debian social contract (see above).

How to ship an FPGA's code

Posted Sep 9, 2004 13:24 UTC (Thu) by forthy (guest, #1525) [Link]

For these blobs, there should at least be an interface to user-land. This
data does not belong to the kernel, and it is not under GPL. It's however
ok to have an interface for the kernel to reach outside, and load these
blobs from files.

This also makes it much easier for vendors and users to update these
files: simply copy it to the right location.

Pointless ideology?

Posted Sep 16, 2004 22:39 UTC (Thu) by job (guest, #670) [Link]

I never understood how end users can suddenly "demand" support for their
favourite hardware from developers.

Look, you don't buy some Mac hardware and expect it to work with your PC.
Why should the situation with Linux be any different? Linux enthusiasts
may very well have more fun things to do than taking it upon themselves
to support every hardware conceivable.

If you buy something, buy from a Linux friendly vendor. Buy Nvidia 3d
cards and you're screwed -- sooner or later. You know that when you buy
it, so don't come whining about it later. There's always the Radeon 9200
if you want something that's actually supported today. This is true even
if you run MS Windows today. Why deny yourself the ability to try Linux
later? Same for winprinters, winmodems, nforce ethernet (supposedly it's
been reverse engineered now though) etc.


Copyright © 2004, Eklektix, Inc.
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds