Obstacles for kdbus

By Jonathan Corbet
April 15, 2015

The kdbus patch set, which adds a D-Bus-like messaging facility to the kernel, has been through several rounds of review over the course of the last year. The number of comments has been dropping with each review cycle, and the code seemed like it could be on track for a relatively easy merge into the 4.1 kernel. A closer look, though, reveals that there was some residual unhappiness from the last rounds that was always likely to flare up into active opposition when an attempt to merge kdbus was made. And, indeed, that is exactly what happened when Greg Kroah-Hartman sent a pull request to Linus on April 13.

This conversation is in full swing as of this writing, so an attempt to fully summarize it would be futile. In brief, though, the complaints take a number of forms. There is unhappiness with the performance of kdbus — a bit surprising, since performance is one of the motivating factors behind this development. There are a number of security-related concerns, especially around how the bus collects and transmits metadata about connected processes. Kdbus is still said to not play well with containers. Some developers find the complexity daunting. And so on.

The core of the disagreement, arguably, can be found in this message from Greg. There, he agreed that the design was "unfortunate" (though he later retracted that statement), and said that kdbus needed to be taken in its current form even if it is not ideal:

D-Bus is a specification that has been out there for over a decade, and we are not designing anything new here, but rather implementing it as designed. We have to be compatible to the existing users of the DBus system, and don't have the luxury of being able to change core things like this and expect the world to be able to change just because the design is not as clean as it should/could be.

Again, just like getting horrid hardware to work properly, sometimes we have to write odd code. Or having to implement a network protocol that doesn't seem to be designed "perfectly", yet is used by a few hundred million systems so we have to remain compatible. This is all that we are doing here for stuff like this.

Remember, this is called kDBUS, not kGENERICIPC, no matter how much we would have liked that to happen from a kernel standpoint. :)

It is probably fair to say that those who are opposed to kdbus in its current form would rather that it were, indeed, kGENERICIPC. They seem to feel that it should be able to support what is needed to implement D-Bus efficiently, but the D-Bus-specific parts, perhaps, should go into user space. After all, there are only so many interprocess communication mechanisms that can be merged into the kernel; the one that goes in, many developers think, should be free of known flaws and should be able to do more than reimplement the D-Bus protocol.

It is hard to say at this point how this discussion will play out or what Linus will decide to do in the end. The chances are good, though, that enough high-profile developers have expressed opposition to derail the merging of kdbus in this development cycle. Complete consensus is not always required to get code into the kernel, but getting code merged when there is serious opposition is still quite hard. This story, it seems, may go on for a while yet.

Index entries for this article
Kernel	kdbus

to post comments

Obstacles for kdbus

Posted Apr 16, 2015 9:11 UTC (Thu) by Thue (guest, #14277) [Link] (3 responses)

> After all, there are only so many interprocess communication mechanisms that can be merged into the kernel

Why do you think there is a limit to how many IPC mechanisms that can be merged? Greg KH has pointed out that kdbus is quite non-intrusive, and that a IPC mechanism of similar complexity is merged about every second kernel release.

kdbus has taken 2 years to implement a given design, so to design and implement a completely new kGENERICIPC would presumably take more than 2 years of they started now. It would make good sense to commit kdbus for use now, rather than forcing kdbus potential users to wait 2+ years.

Obstacles for kdbus

Posted Apr 16, 2015 18:34 UTC (Thu) by daniels (subscriber, #16193) [Link]

Or to design and implement the old kGENERICIPC, or as it was called at the time, AF_BUS: https://lwn.net/Articles/504970/

Unfortunately the response from David Miller (the net/ maintainer) was that he was going to ignore the series without even looking at it (he did), because he felt multicast was sufficent (it wasn't). And here we are.

Obstacles for kdbus

Posted Apr 23, 2015 17:14 UTC (Thu) by ksandstr (guest, #60862) [Link] (1 responses)

As an educated guess, one of the reasons is that a half-decent IPC mechanism (i.e. one that provides something over AF_UNIX, SysV IPC, POSIX IPC, and ones backed with them) would tie into the process scheduler to minimize round-trip latency and cache effects. The one in Linux has been a mishmash of jumbles since the scheduler wars of about a decade ago, making alterations a subtle affair prone to hidden regressions, rightly feared by most.

It seems to follow that kdbus (and perhaps Android's "binder" as well) would either integrate elbow-deep with the scheduler, be using a general interface shared between the fresh-faced new IPC mechanisms to that effect, or be nothing special and therefore unworthy. My two Internet cents rest on the lattermost hypothesis' eyelids.

The unhappiness about speed is quite interesting. Was kdbus developed without a way to measure performance in comparison to the user-space solution, one of its stated & well-publicized design requirements? Surely such critique is the easiest to overcome: just slap down "my first rdtsc" and iterate until it's any good.

Moreover there are several Free Software super-scalable message-queuing frameworks kicking around in the enterprise middleware space right now; for example 0mq and its descendants-in-spirit. These seem to do just fine even without a kernel-side bypass lane. Why is that? Why do parallel computing interfaces like MPI manage full utilization of both local and clustered resources without going into the kernel, but dbus must pierce that veil?

Far be it from me not to suspect that kdbus is a way to cement the systemd takeover of GNU/Linux userspace. These pro-kdbus arguments of inevitability, undifferentiated from those for old GNOME's Bonobo/ORBit, and similes to "buggy hardware" which software by definition isn't, seem vacuous and technically inept like those used to back systemd in the first place; and the absence of concrete advantage beyond "it'll be over as soon as you stop resisting" just makes it seem all the more consciously wack.

Obstacles for kdbus

Posted Apr 23, 2015 21:38 UTC (Thu) by Cyberax (✭ supporter ✭, #52523) [Link]

> Why do parallel computing interfaces like MPI manage full utilization of both local and clustered resources without going into the kernel, but dbus must pierce that veil?
MPI systems rarely use a 'dialog' pattern of communications where one process sends a message to another process and then awaits the result. Exactly because it's so extremely inefficient without something like binder/kdbus.

Obstacles for kdbus

Posted Apr 16, 2015 11:24 UTC (Thu) by johannbg (guest, #65743) [Link] (19 responses)

The obstacles for kdbus are Andy Lutomirski and Eric W. Biederman.

If you put aside the way too many tongue lashings then technical criticism that has taken place in response to Greg ( which says more about the kernel community abusiveness than about kdbus design choices ), it's going to be interesting to see if the kernel community deals with commits on pure technical level or based on peoples personal "feelings" because at this point it has become quite clear that it matters not how many times they rewrite at the ackers whim, it still will be nacked.

Heck there was even attempt to use stall tactics to delay the inclusion of this further through "meet and greet discussions" ( one would have thought past two years had provided enough opportunity to discuss this at various places )

Fortunately James Bottomley appeared and killed that attempt in birth in that thread via common sense.

So far it looks like Al Viro is the only one that actually bothered to read and review the code and as a result of that provide feedback of things he liked to see improved ( after his usually the kernel is a pile of shit rants and everything for inclusion in it too ).

He actually manage to give Greg and the rest of the guys something tangible work with and fix unlike the rest of the responders who are to busy feeding Greg bullshit .

Those responding expert individuals that trust nobody but themselves and are hoping for vague glorified future how this should be handled, can step up of their iron thrones and start writing that future on their rainbow farting pony's in the next coming years but until that future arrives and everyone will be bathed in it's unpreceded functionally glorified fairy dust, kdbus has to make do since it's built upon the experience and the input of the people that actually have been working with this stuff for what past 10 years.

So seriously shut up and commit that code already this is getting ridiculous...

Obstacles for kdbus

Posted Apr 16, 2015 13:17 UTC (Thu) by corbet (editor, #1) [Link] (16 responses)

I'll resist the temptation to address everything in your comment, but somebody has to say that you are doing a real disservice to Eric and (especially) Andy. There's nobody else who has put in as much time as they have going through the code, making suggestions, and causing it to get better. Kdbus is a much improved patch set as a result of their work. You may disagree with their conclusions, but it is wrong to cast them as mindless obstructionism in order to ignore them.

(In case you wonder, I'm in favor of merging kdbus, or something that can implement that functionality, but I'm also in favor of doing it right).

Obstacles for kdbus

Posted Apr 16, 2015 15:26 UTC (Thu) by Thue (guest, #14277) [Link] (2 responses)

I don't know about those two in particular, but parts of that kernel thread seemed to descend into anti-systemd circlejerking. There does seem to be an unreasonable amount of negativity just because of the association with systemd.

Obstacles for kdbus

Posted Apr 16, 2015 16:01 UTC (Thu) by aigarius (subscriber, #7329) [Link]

The only thing related to systemd in that thread is that systemd is quite sure to be using kdbus and all its functionality and is likely to require it as soon as it is mainlained, so it was disingenuous to dismiss API design criticism on the basis that "it is all optional anyway". That is a reasonable assumption and not some kind of anti-systemd hate.

Obstacles for kdbus

Posted Apr 16, 2015 18:37 UTC (Thu) by daniels (subscriber, #16193) [Link]

Yes, a lot of that thread was unnecessary derailing in the form of 'this is unnecessary because I don't like D-Bus or systemd and let me explain at you why I don't'. I didn't see that coming from Eric and Andy though.

Obstacles for kdbus

Posted Apr 16, 2015 16:14 UTC (Thu) by viro (subscriber, #7872) [Link] (9 responses)

... to which I would like to add that I do _not_ appreciate the way my involvement had been misrepresented. Recap for those who'd missed the show:

various people: observe that there had been zero reviewed-by
greg: what can I do? Can't make 'em review it...
me: you might start with getting it into a shape that would be somewhat easier to review; as it is, reviewers get not just to verify that your actual locking matches the claimed one (which would be perfectly reasonable thing to expect of them), they get to _deduce_ the locking rules. In a piece of code with fairly complex locking and no locking-related documentation whatsoever. Moreover, locking doesn't look as if such a documentation had been written - >this< and >this< would stick out as sore thumbs. What are the locking rules for <this> and <that>? Document your damn locking if you want reviews - sure, I can spend a week and try to document it, but it'd be better if you guys did that[1]
greg: fair enough, >this< and >this< don't look good and locking docs are needed. We'll do that. [No answers to specific questions on locking order are given].
... and later on lwn, a thoroughly dishonest advocate in <grandparent of this posting> had the gall to praise me for being helpful (as opposed to Eric and Andy) *and* concluded with "shut up and commit it already". Completely ignoring the fact that neither documentation needed for reviewing it, nor, as the result, any new code reviews have materialized.

For the record: what I'd done does _NOT_ qualify as code review. I have told Greg what was missing for doing any meaningful review. He appeared to have agreed that it was missing and would be added, but so far it hadn't been done - not even on the level of "this is supposed to nest inside <this and that> and outside <this, this and that>; we'll do more detailed text later, but that at least ought to cover your specific questions". I'm NOT taking a week-long break smack in the middle of merge window to waste on what they could do in a few hours. And that information is a prerequisite for any meaningful review, as far as I'm concerned. It's not an "oh, we'll do it after the thing gets merged" kind of thing - once it's in, it's in and if a later review finds out that there _are_ locking problems in there, we won't be able to touch the user-visible interfaces in process of fixing those.

To Johann: you, sir, are an outstanding example of variability of our species. worthy of any curio cabinet, whatever the formalin might cost.

[1] incidentally, I'm not 100% sure, but it smells like some of the ordering can't be deduced from the call graph alone - there are functions with or'ed flags among arguments, taking different locks depending on those flags and called in varying locking environments, so one would need to examine the conditions needed for them to take given lock, then figure out which call sites satisfy those and look at the locking environment for those; great fun to be had, in other words. For them that task is greatly simplified since they know what the locking order should be (assuming they do know that, of course). The difference is between proving that code satisfies given property and figuring out what properties does it satisfy; the latter obviously can get much harder if the code is complex enough. And that it is... Authors are really at very big advantage when it comes to producing such thing.

Obstacles for kdbus

Posted Apr 16, 2015 17:08 UTC (Thu) by johannbg (guest, #65743) [Link] (8 responses)

I think curio cabinet ( me in it or not ) would indeed be a perfect match for the glass house you reside in.

Obstacles for kdbus

Posted Apr 16, 2015 17:38 UTC (Thu) by viro (subscriber, #7872) [Link] (7 responses)

Umm... Usually references to glass houses are meant to be read as an accusation of hypocrisy. Care to elaborate, filth?

Obstacles for kdbus

Posted Apr 16, 2015 18:00 UTC (Thu) by johannbg (guest, #65743) [Link] (6 responses)

Last time I made reference to you living in a glass house you did not have the mental capacity to understand it so I sincerely doubt that your slowly growing senile brain can comprehend it now.

Since you aint getting any younger you might want consider stop sitting in there throwing bricks, hoping that someone dies and makes you the god you believe yourself to be while sitting on an iron throne playing game of moans and make better use of those limbs like wiping your own ass while you still can.

Feel free to seal those lips while your at it.

Obstacles for kdbus

Posted Apr 16, 2015 18:22 UTC (Thu) by viro (subscriber, #7872) [Link] (5 responses)

Learn to flame. The only insult in that incoherent pile of metaphors is the fact that we belong to the same species. And since you cause _that_ offence simply by being yourself, well... everyone got used to it by now. Sad...

Obstacles for kdbus

Posted Apr 16, 2015 19:05 UTC (Thu) by johannbg (guest, #65743) [Link] (2 responses)

I guess they teach how to flame next to the kernel power tripping and be disrespectful to one another classes people take when they join the kernel community.

Yeah sure I might be a sad excuse for a human being ( someone needs to bear that title so it can just as well be me I'm no better or worse then the next guy ) however what is truly pathetic is the waste of words that took place when I actually gave you an complement and you responded the way you did.

If I would have known how much of an huge asshole you are then I never would have bothered and would have known forehand tearing you a new one is a waste of time since it's mission impossible. Things dont get bigger than this.

The lesson I learned from this was that I should have listened to people but I usually want to first hand experience people before I judge them or label them.

Obstacles for kdbus

Posted Apr 16, 2015 19:17 UTC (Thu) by jake (editor, #205) [Link]

this has no place here ... if you two want to call each other names, please do it elsewhere.

thanks,

jake

Obstacles for kdbus

Posted Apr 16, 2015 19:25 UTC (Thu) by viro (subscriber, #7872) [Link]

Marginally better (at least this time it was a coherent text), but jake is correct - lwn is not the place for that kind of lessons. If you want to continue this, er, conversation - you know my email address.

Obstacles for kdbus

Posted Apr 23, 2015 13:45 UTC (Thu) by rich0 (guest, #55509) [Link] (1 responses)

And we wonder why people think the kernel development team behaves non-professionally...

Obstacles for kdbus

Posted Apr 25, 2015 13:10 UTC (Sat) by Kamilion (subscriber, #42576) [Link]

Because they *DO*, and it is quite entertaining to us in the peanut gallery.

It also shows us they are human beings; not monotonic code robots.

It's not like there is any *requirement* in the linux community to be professional or to be *A* professional. It happens to be a guideline; something that at least some people will strive for.

I could deliver a rant about how social media is rotting everyone's brain and how IRC should be the model everyone should strive for; but I'm sure there'd be someone out there complaining that they couldn't see cat pictures or whatever Terry had for lunch, or the latest vine viral video of people playing with action figures. And then the inevitable comment that "oh, IRCCloud will embed link it finds in the scrollback." proving that it can be as simple as changing a client to meet modern user expectations, not a protocol.

Obstacles for kdbus

Posted Apr 17, 2015 9:46 UTC (Fri) by sergey.senozhatsky (subscriber, #91933) [Link] (2 responses)

I sedond this. kdbus is still getting better after all. compare it to kdbus that greg pushed to staging 1 year ago.
all I'm saying is -- no, it's not "anti-systemd only".

Obstacles for kdbus

Posted Apr 17, 2015 13:33 UTC (Fri) by gregkh (subscriber, #8) [Link] (1 responses)

> compare it to kdbus that greg pushed to staging 1 year ago.

I never pushed any kdbus code to staging, are you thinking of something else?

Obstacles for kdbus

Posted Apr 17, 2015 17:16 UTC (Fri) by sergey.senozhatsky (subscriber, #91933) [Link]

> I never pushed any kdbus code to staging, are you thinking of something else?

oh. I was pretty sure that for some very short period of time kdbus was in staging; in 2013 may be. what was it... ktap? hm, so I was looking on kdbus on github back then.

yes, I was wrong. sorry.

Obstacles for kdbus

Posted Apr 16, 2015 22:25 UTC (Thu) by asaz989 (guest, #67798) [Link]

Hmmm. I'm new to this whole discussion. So I might not be correctly understanding how someone who complains about "kernel community abusiveness" then gets off flaming people about how they "can step up of [sic] their iron thrones and start writing that future on their rainbow farting pony's [sic]".

I much prefer Linus's brand of abusiveness, which is at least unapologetic and cites its sources.

Obstacles for kdbus

Posted Apr 23, 2015 7:19 UTC (Thu) by oldtomas (guest, #72579) [Link]

Quoth the poster:

> If you put aside the way too many tongue lashings then technical criticism that has taken place in response to Greg ( which says more about the kernel community abusiveness than about kdbus design choices )

And then:

> Those responding expert individuals that trust nobody but themselves [...] can step up of their iron thrones [...] rainbow farting pony's [...] everyone will be bathed in it's [...] fairy dust [...]

Conclusio:

> So seriously shut up and commit that code already this is getting ridiculous...

Oh, goody. You have convinced me now.

Seriously: if we don't learn (and I mean ALL sides) to dial down our egos a bit and just accept that the others aren't idiots either, this is going to be less and less fun.

Obstacles for kdbus

Posted Apr 16, 2015 12:10 UTC (Thu) by aigarius (subscriber, #7329) [Link] (20 responses)

The thing is that as soon as it is merged into kernel, whatever design choices there are, are automatically part of the kernel ABI that will have to be maintained for a long time. The largest objections are for the metadata attached to messages which exposes huge array of kernel internal data structures as part of this ABI. Which would then mean that these structures may no longer change or they would break userspace. That *is* intrusive.

Arguments about optionality of kdbus do not hold water as systemd is basically everywhere now and as it is expected to use kdbus exclusively as soon as it is merged.

Instead of thinking from the perspective of maintenance of unchanging kernel ABI, kdbus developers tend to do a minimum effort port of DBUS. It should be very well possible to address the metadata issues while actually improving the design. Off the top of my head - most of the included metadata is only actually used for logging. Why not separate the logging data stream from the actual data? Then you can have more useful permission controls on that data and only provide the loggable metadata where it is needed - to the logger daemon. And a logger daemon can be adapted to keep working when kernel metadata changes, thus removing huge part of the ABI objections.

You can't just take something from userspace and jam it into the kernel. It needs to be written based on the same rules and assumptions as the rest of the kernel code.

Obstacles for kdbus

Posted Apr 16, 2015 12:29 UTC (Thu) by quintesse (guest, #14569) [Link] (1 responses)

> most of the included metadata is only actually used for logging

AFAIU the reason for adding the meta-data was to have access to it in a race-free manner, that the user-space is going to need that data anyway but if it has to go look for it itself the need information might already have changed/disappeared.

What I don't understand is why these meta-data structures couldn't change saying that otherwise user-space would break. If user-space is already using that information today how does it not break when they change those structures in the kernel?

Obstacles for kdbus

Posted Apr 16, 2015 14:29 UTC (Thu) by drag (guest, #31333) [Link]

> If user-space is already using that information today how does it not break when they change those structures in the kernel?

Because it wouldn't be a kernel ABI change that would be breaking them.

In Linux userspace breaks userspace all the time. It's extremely unstable. That is nothing new... but if you get bad design in the kernel then that bad design is going to be there practically forever.

So if there is a fundamental design flaw in dbus/kdbus it better get solved before merging into the kernel.

Obstacles for kdbus

Posted Apr 16, 2015 12:47 UTC (Thu) by tomegun (subscriber, #56697) [Link] (1 responses)

> The largest objections are for the metadata attached to messages which exposes huge array of kernel internal data structures as part of this ABI.

kdbus does not expose anything that is not already exposed to userspace, it is simply making sure the exposed data can be retrieved in a race-free manner.

Obstacles for kdbus

Posted Apr 16, 2015 12:56 UTC (Thu) by tao (subscriber, #17563) [Link]

Now, now, don't go spoiling a perfectly fine flame by introducing facts.

Obstacles for kdbus

Posted Apr 16, 2015 13:05 UTC (Thu) by mezcalero (subscriber, #45103) [Link] (14 responses)

A "huge array of kernel internal data"? That's certainly not the case. None of the data kdbus optionally attaches as metadata to bus calls is "internal". All of it is available thorugh /proc as well. The only reason we attach it to messages is to fix races around retrieving them, since we need some of the parts for doing authentication and security relevant logging.

The meta-data is used for authentication, logging and introspection purposes, not just "logging". Of the various fields we provide, some are useful for authentication, and others for introspection, and others for logging, some for all three cases, others for two cases. It really depends.

And the "logging data stream" should not be separated out, since we provide it so that the recipient of a method call can properly log about the method call it is executing, about what it is doing for it, and on which behalf it is doing it, all in a race-free fashion, in one log record. If you however split things into multiple streams, you will have difficulty matching up the events from the streams, and things will be necessarily raceful.

Lennart

Obstacles for kdbus

Posted Apr 17, 2015 0:23 UTC (Fri) by BenHutchings (subscriber, #37955) [Link] (8 responses)

All of it is available thorugh /proc as well.

Not with hidepid enabled, it isn't.

Obstacles for kdbus

Posted Apr 17, 2015 10:23 UTC (Fri) by mezcalero (subscriber, #45103) [Link] (7 responses)

Well, "hidepid" doesn't work really. Userspace cannot really cope.

That said, kdbus actually has a feature similar to hidepid: you can set a mask for all creds via /proc or even kernel cmdline, and it has been added on request of Andy's.

Lennart

Obstacles for kdbus

Posted Apr 17, 2015 12:01 UTC (Fri) by spender (guest, #23067) [Link] (6 responses)

It worked just fine for a decade and a half in various forms before you showed up. Everyone else has to cope with your code, but not the other way around -- is that how it works?

-Brad

Obstacles for kdbus

Posted Apr 20, 2015 16:15 UTC (Mon) by mezcalero (subscriber, #45103) [Link] (5 responses)

Oh, it was never useful in general purpose operating systems. Too many programs need data from /proc. I mean, it already falls apart if you want to run an OS that uses polkit really...

I'd actually love to support it nicely in systemd. For example, I think a per-service setting "HideProc=" would make a ton of sense, that could be used to set it for individual services, by combining it with file system namespaces (so that each service gets his own /proc instance with the appropriate setting). However, this is currently not possible, due to the weird way it is implemented: it pretends to be a mount option, but actually doesn't apply to each mount individually but is a global, kernel-wide option.

Would love to see this fixed, and if so I'd be happy to expose this in systemd to make it useful for everybody without breaking things that cannot cope with it.

Lennart

Obstacles for kdbus

Posted Apr 20, 2015 21:55 UTC (Mon) by spender (guest, #23067) [Link]

Sorry, I should have clarified: until you and the braindead security modeling of fd.o came around, of which polkit is a perfect example. It's no wonder that when talking about a feature designed primarily for multi-user systems, you point out that it's incompatible with the insecure-by-design polkit, software that no sane server administrator would have installed.

-Brad

Obstacles for kdbus

Posted Apr 22, 2015 16:05 UTC (Wed) by intgr (subscriber, #39733) [Link] (2 responses)

Spender happens to be correct here. I was successfully using hidepid=2 on all my desktops until a certain update. It's not clear where the blame lies here, it may well be due to the cgroups design in the kernel, but systemd-logind was the immediate cause. It broke because logind assumes that /proc/1/ is always there and refers to the init process: http://lists.freedesktop.org/archives/systemd-devel/2012-...

Obstacles for kdbus

Posted Apr 23, 2015 16:52 UTC (Thu) by thestinger (guest, #91827) [Link] (1 responses)

It's still easy to get it working. You need to create a proc group and use hidepid=2,gid=$PROC_GID and then add the polkit user and systemd-logind service to it:

% cat /usr/lib/systemd/system/systemd-logind.service.d/hidepid.conf
[Service]
SupplementaryGroups=proc
% grep proc /etc/group
proc:x:26:polkitd

Obstacles for kdbus

Posted Apr 24, 2015 9:56 UTC (Fri) by intgr (subscriber, #39733) [Link]

Thanks! I have documented this for other users at https://wiki.archlinux.org/index.php/Security#hidepid

Obstacles for kdbus

Posted Apr 23, 2015 16:42 UTC (Thu) by thestinger (guest, #91827) [Link]

You can already use hidepid=2,gid=xxx and then set exceptions for services by adding them to the appropriate proc group. There are very few exceptions required for a typical desktop system (polkit and systemd-logind, but only because systemd-logind drops some capabilities). I have it working fine on Android where there's more work required.

Obstacles for kdbus

Posted Apr 17, 2015 9:54 UTC (Fri) by aigarius (subscriber, #7329) [Link] (4 responses)

I understand how it is more convenient to just ask the kernel to attach all the possible metadata and then, on the receiver side, decide which of it you need and how to use that. And how that removes the race conditions. That is a very good thing and a key to why kdbus is needed. On the other hand it is also understandable how the kernel developers want to minimise the outgoing information to ensure higher future flexibility.

The logging issues with a separate stream can be solved, for example by attaching a unique message id that is provided both on the log stream and to the receiver. It would be perfectly fine to see "message 123 passed from process xxx (uid:1002, ...) to process yyy (uid: 500, ...), ..." as a log entry from kdbus and then "error processing message 123: barfooed" from the receiver.

I mean - does the receiver even *need* to know the pid of the sender? Would it not make more sense to have some filter in the kernel, so that only messages from processes that actually have some capability ever reach the receiver? With an established user base of dbus it should be possible to actually find out what filters people are using and what metadata actually is critical for the receivers.

The existing security model of dbus as described in http://article.gmane.org/gmane.linux.kernel/1930451 is at best confusing. That is a clear example of something that needs to change in how dbus works and should have been changed even without the push to kbus. And once it is clear that kdbus *must* be different from dbus, then you can easily find other places to improve and clean up the protocol as well as ways to paper over those changes in libdbus or by providing a new, more convenient userspace library.

I am, admittedly, nowhere near an expert on this, but I do prefer the experts to come together with an open mind on the requirements of all sides. Without accusations of personal attacks.

Obstacles for kdbus

Posted Apr 17, 2015 12:26 UTC (Fri) by pizza (subscriber, #46) [Link]

> The existing security model of dbus as described in http://article.gmane.org/gmane.linux.kernel/1930451 is at best confusing. That is a clear example of something that needs to change in how dbus works and should have been changed even without the push to kbus

And that is a non-starter, because DBUS is an established standard that has been in use for nearly a decade.

Obstacles for kdbus

Posted Apr 18, 2015 11:09 UTC (Sat) by mathstuf (subscriber, #69389) [Link] (2 responses)

> I mean - does the receiver even *need* to know the pid of the sender?

Yes. I'm implementing the SecretService API in keepassx and I'm locking it down so that access is based on a per-application basis (so a malicious app can't rummage around once it is open). Determining the binary is dependent on the executable pointed to by /process/PID/exe. I'd be grateful for better solutions, but that's the best I can do right now. I'm pretty sure it isn't subject to PID races since the reply won't be hooked up if it gets replaced (though exec could happen I suppose).

Obstacles for kdbus

Posted Apr 18, 2015 18:44 UTC (Sat) by nybble41 (subscriber, #55106) [Link] (1 responses)

> ... I'm locking it down so that access is based on a per-application basis.... Determining the binary is dependent on the executable pointed to by /process/PID/exe.

Unless you're doing something to prevent LD_PRELOAD and ptrace(), I'm not sure what you intend to accomplish by this. Even ignoring any potential PID race conditions, the code which is running and has access to the message bus is not determined solely by /proc/PID/exe.

Obstacles for kdbus

Posted Apr 18, 2015 19:11 UTC (Sat) by mathstuf (subscriber, #69389) [Link]

Currently, kwallet and gnome-keyring have a single "unlock" state at the collection level (though ISTR kwallet also gating based on the app, access was still global once permitted). Once, say, Firefox unlocks the store, any app can go in and query passwords. To me, it doesn't make sense to allow Firefox access to passwords intended for Pidgin or git, and vice versa (and they can even delete each other's passwords). As for the actual code running, yeah, something better would be nice, but it's more to minimize leaking if an application does have problems. It's already a pain to migrate dozens of accounts to a password store in the first place and I don't want to imagine trying to get back once malware starts targeting password stores (either by quietly siphoning, deleting, or modifying passwords in them) through the apps which communicate with them.

Obstacles for kdbus

Posted Apr 16, 2015 13:07 UTC (Thu) by alexl (guest, #19068) [Link]

How is this a minimum effort? Its the third effort of making a kernel dbus implementation, and this version has gone through large changes based on upstream feedback.

Obstacles for kdbus

Posted Apr 16, 2015 15:44 UTC (Thu) by dashesy (guest, #74652) [Link] (2 responses)

Cannot resist but to ask, why not have both, first a `kGENERICIPC` then use it for `kdbus`? If all I want is an improved IPC I can use the first without the dance needed for D-BUS, that will make embedded folks happier too.

Obstacles for kdbus

Posted Apr 16, 2015 18:11 UTC (Thu) by drag (guest, #31333) [Link]

Linux definitely needs something dbus-like that is fast and universal. Application developers definitely want it and it will have no shortage of users once kbdus-like is integrated into the kernel. It's been something that has been needed for a long time now.

So if you do get 'kgenericipc' in that will not really relieve the need/desire for kdbus-like functionality. Then you will just end up with 2 sets of API/ABIs to maintain instead of just one.

This is why it's so critical to get it right. It's going to be used almost immediately by a huge number of different things. If it was to get in now you'll see distributions shipping significant amount of software that requires it by the end of the year. If it's wrong now there is never going to be any chance to fix it.

On the upside I am willing to be that a lot of it's 'crapiness' is due to the fact that it's initial design has been fine tuned and fixed over the years and what often appears to be kludges is just a sign of a maturity.

Obstacles for kdbus

Posted Apr 23, 2015 13:48 UTC (Thu) by rich0 (guest, #55509) [Link]

I think the objection is that nobody wants to write it, especially those asking for it. The kdbus proponents say that they already have a solution that they want and which all their userspace consumers want. The generic alternative is something nobody actually intends to use, and nobody actually wants to write.

Obstacles for kdbus

Posted Apr 16, 2015 17:21 UTC (Thu) by flussence (guest, #85566) [Link] (1 responses)

Rather than beating D-Bus to make it go faster, I'm beginning to think it'd be a more efficient use of effort to convince application developers to not use it in the first place. It's starting to look a lot like the ActiveX situation all over again.

Obstacles for kdbus

Posted Apr 16, 2015 18:43 UTC (Thu) by daniels (subscriber, #16193) [Link]

> Rather than beating D-Bus to make it go faster, I'm beginning to think it'd be a more efficient use of effort to convince application developers to not use it in the first place. It's starting to look a lot like the ActiveX situation all over again.

A lot of people have said this over the past 12 years, and yet no viable replacement has emerged.

Havoc wrote an extremely eloquent mail explaining why the model was as it was: because applications a) did not want to use a more complex model requiring them to manage their own state, and b) were likely to get such a model wrong, resulting in security flaws among others.

Any replacement for D-Bus must therefore retain many of its same properties, and that no such replacement has emerged (with the criticisms largely being of its design goals), probably says quite a bit about its quality.

Obstacles for kdbus

Posted Apr 19, 2015 8:54 UTC (Sun) by hrogge (guest, #100012) [Link] (4 responses)

The problem with the "capabilities situation" is really hard to solve in a clean way if an userspace daemon wants to rely on kernel capabilities to check if another process is allowed to do a certain action.

There is no really good way to check if you have the capabilities for a kernel syscall except for just calling it. But this doesn't work well if you have to do some userspace work before you call it, which is especially important for things like a reboot.

I wonder if anyone has an idea how to resolve this issue, how can one process prove it is allowed to call a certain Linux syscall to another process without calling it?

Obstacles for kdbus

Posted Apr 19, 2015 15:26 UTC (Sun) by flussence (guest, #85566) [Link] (3 responses)

> I wonder if anyone has an idea how to resolve this issue, how can one process prove it is allowed to call a certain Linux syscall to another process without calling it?

Easy. Remove it. Throw all of this broken second-system nightmare out the window, and all its workarounds, and all their workarounds too. Let's go *forward* to what actually worked before all this fd.o crap came along.

In the old days, distros allowed non-root users to do scary and evil things like shut down, mount USB sticks and generally use their own computer by way of setuid-root or setcap helper binaries, or background services talking over a socket — those didn't need you to pass through a digital TSA checkpoint before they'd talk to you. In this savage and primitive culture there was no PolicyKit, or ConsoleKit, or UKit, or KitKitFactoryFactory, or libxml, or a Javascript VM, or HALd, or udevd, or SELinux, or dbus. Did the world end for lack of those? Hardly, the OS was *more* usable and *more* secure than it is today!

To those who would cry "we need all this": you're lying and you know it.

Obstacles for kdbus

Posted Apr 19, 2015 16:37 UTC (Sun) by mpr22 (subscriber, #60784) [Link] (2 responses)

A bold call to arms! Who are you suggesting do the work you think should be done?

Obstacles for kdbus

Posted Apr 20, 2015 16:58 UTC (Mon) by flussence (guest, #85566) [Link] (1 responses)

You're making the false assumption that this would require an increase in work.

Obstacles for kdbus

Posted Apr 20, 2015 17:31 UTC (Mon) by mpr22 (subscriber, #60784) [Link]

When suggesting other people do work, it is not only the quantity that matters.

kdbus vs binder (Android IPC)

Posted Apr 21, 2015 8:43 UTC (Tue) by debacle (subscriber, #7114) [Link] (1 responses)

Many believe, that binder is far more problematic than kdbus, yet it can go into the kernel. I wonder, why kdbus is treated differently than binder?

kdbus vs binder (Android IPC)

Posted Apr 22, 2015 11:07 UTC (Wed) by tbird20d (subscriber, #1901) [Link]

Because if binder breaks, Google has to fix it, and no one else cares. For non-Android, binder is "use at your own risk". The ability of the kernel developers to ignore Android is actually beneficial in this case.

Obstacles for kdbus

Posted May 3, 2015 17:57 UTC (Sun) by oak (guest, #2786) [Link]

I remember looking at D-BUS performance on ARM for small messages, ~8 years ago. I wonder whether and how the issues from back then have been nowadays fixed:

* Worst system performance issue was daemon memory consumption & fragmentation in case when clients didn't read (frequently sent) messages that they had subscribed for. Daemon just had to buffer them until it ran out of memory. There was no API/setting that would have allowed daemon to discard unread messages (messages were about status changes so only last one was relevant). Suspending message sender wasn't an option because there were also other, responsive receivers for those messages.

* For message transfer performance, daemon implementation was an issue, not it being in user-space. It was CPU bound because daemon internally marshaled / demarshaled messages too many times.

* Latency was mainly a function of socket buffer size, with small buffers you got faster turn-around (faster scheduling), with larger buffers, higher throughput.

How kdbus handles the first case?