Wayland - Beyond X (The H)

Posted Feb 14, 2012 22:12 UTC (Tue) by farnz (subscriber, #17727)
In reply to: Wayland - Beyond X (The H) by rqosa
Parent article: Wayland - Beyond X (The H)

That's basically what X on Wayland is - another way to look at Wayland is that it's the construction kit to build such an X server, and as we're putting all the bits together, we might as well expose the lower layers so that you can bypass X if you want to.

Wayland's protocol is a spiritual subset of X11 - it's basically clipboard data exchange, input and DRI2, without the rest of the X11 bits. The trouble with building this atop X11 is that the interactions between X11 core protocol and the subset that Wayland provides are nasty (for example, X11 has traditional input events that "know" that you only ever have one mouse position and a set of buttons - what is the "correct" mouse position if you're doing multitouch input? Where did you touch when you make the "zoom in" gesture - the start point? The end point? - noting that "neither" and "both" aren't acceptable answers to core X input), and you have to get them right to truly claim to be "an X server with built-in compositing". The rules around subwindows are another example

Wayland takes it in a different direction - throw away the bits that are hard to implement yet rarely used, and make a Wayland client (the X server) implement them if they're wanted. If network transparency really is a critical feature, someone will step up to implement it - whether by doing the work themselves, or by paying someone such as Red Hat to include it.

to post comments

Wayland - Beyond X (The H)

Posted Feb 14, 2012 22:24 UTC (Tue) by dlang (guest, #313) [Link] (3 responses)

I understand that X supports multiple mice on a screen, and I've seen stuff recently about it supporting multitouch as well.

Wayland - Beyond X (The H)

Posted Feb 14, 2012 23:11 UTC (Tue) by farnz (subscriber, #17727) [Link] (2 responses)

It has some deep seated issues around the way multiple mice and multitouch interact with "traditional" X clients - if we could just assume that all X applications will be rewritten to assume multitouch support, we could get rid of a decent collection of interesting special cases.

There are at least three cases that XI2 just can't get right when interacting with traditional X clients:

When I have multiple simultaneous touch points, where is the "correct" location for the emulated single touch? There's no right answer for this - it could be one or more of the individual touch points, or it could be the position of the cursor, or it could be an average of the locations of the touch points. Exactly which one is right is a function of the underlying UI, which X can't see into.
If the user does a "global" gesture, you need to tell the client "I know I've been feeding you motion and touch events, but actually, they've turned out to be a global gesture. You need to forget that they happened". The alternative is to not send the events at all until touch up - painful if you're trying to drag an icon.
With multiple input devices, how do I synthesise the single mouse that a traditional X client expects? Whatever I do is going to fail in some corner cases - either there will be button events that don't correspond to your expectations (e.g. because I simply serialised two mice worth of events into one stream, and mouse one was trying to drag an icon when mouse 2 came in and double-clicked a different icon), or there will be a lack of events when you expected some (e.g. because mouse 1 is lurking over a window, but you've moved mouse 2 over it as well, and have forgotten about mouse 1).

Wayland - Beyond X (The H)

Posted Feb 14, 2012 23:38 UTC (Tue) by khc (guest, #45209) [Link] (1 responses)

Curious comment from a bystander, how does wayland plan to address these issues? Specifically 1) and 2).

Wayland - Beyond X (The H)

Posted Feb 15, 2012 12:15 UTC (Wed) by farnz (subscriber, #17727) [Link]

Wayland's way of addressing them is simple - it doesn't have a historic protocol to worry about, so it can assume that you are multitouch aware. It handles them the same way XI2 does when dealing with XI2 clients.

Specifically addressing the three points I made earlier:

When you have multiple touch points, Wayland reports all the touch points individually, including their locations, and the client is expected to work out what that means. For example, it could highlight a button if any of the touch points are within that button, if that's appropriate for this client.
Touch event streams are terminated with either a "touch complete - go process" or "actually, that was the beginning of a global gesture - ignore it" event. This lets applications provide feedback to the user as the touch happens (e.g. drag pointer instead of normal pointer), but delay taking action until the touch completes.
You get events from each input device individually, so the client can distinguish event streams from multiple mice.

Wayland - Beyond X (The H)

Posted Feb 15, 2012 10:36 UTC (Wed) by renox (guest, #23785) [Link] (8 responses)

> Wayland's protocol is a spiritual subset of X11

The issue is at which point a subset is good enough?

> If network transparency really is a critical feature, someone will step up to implement it

Which network transparency?
Inefficient network transparency based on buffer would be easy to add sure, but efficient network transparency to Wayland would need to change a lot Wayland, adding the equivalent of XRender in the API (especially the part used for text rendering) so that toolkits would need to be significantly changed also (may not be too difficult actually for a multi-platform toolkit which can also use X11, but for a Wayland-only toolkit..).

Wayland - Beyond X (The H)

Posted Feb 15, 2012 12:25 UTC (Wed) by farnz (subscriber, #17727) [Link] (7 responses)

So, the expected path is that Wayland handles local compositing and input. There will probably be a VNC-alike inefficient remoting protocol for native Wayland at some point, assuming anyone gets interested (which is a safe assumption, as VNC exists for platforms like Windows that didn't have a native remoting protocol from day 1).

Something else, toolkit specific since that's the layer at which you have semantic information to do a good job, will handle efficient network transparency. You can already do this today for GTK+ by writing a new GDK backend that produces a network stream that can be fed back into GDK at the other end.

And note that (because Wayland's protocol is basically a subset of X11's) it's entirely possible already to produce an X11 toolkit that cannot be efficiently transported over the network. I've already encountered proprietary X11 applications whose custom toolkits handle X11 by sending repeated full-window bitmaps over the wire, after all, so it's not like X11 is a guarantee that you'll do something network-efficient.

Wayland - Beyond X (The H)

Posted Feb 15, 2012 13:54 UTC (Wed) by renox (guest, #23785) [Link] (5 responses)

> Something else, toolkit specific since that's the layer at which you have semantic information to do a good job

Toolkits have the information yes, but hopefully they can use a generic protocol (say Wayland 2.0 with something like Render added), not a toolkit specific protocol: it would be a compatiblity nightmare..

Wayland - Beyond X (The H)

Posted Feb 15, 2012 15:29 UTC (Wed) by farnz (subscriber, #17727) [Link] (4 responses)

The problem right now is that we don't actually know what that generic protocol should look like. XRender is better than core X11, but is still neither good enough for modern toolkits (which are moving to OpenGL to escape the problems with XRender), nor as efficient as something like NeWS was.

Hopefully (yes, I'm an optimist), we will see some experimentation with remoting protocols as a fallout from Wayland, and eventually find a better compromise than "accept the limitations of X11".

Wayland - Beyond X (The H)

Posted Feb 16, 2012 13:08 UTC (Thu) by renox (guest, #23785) [Link] (3 responses)

> XRender is better than core X11, but is still neither good enough for modern toolkits (which are moving to OpenGL to escape the problems with XRender)
Which issue are you talking about? The speed of the rendering?
I expect that any protocol which is network efficient (WAN) will reduce the rendering speed, it's a tradeoff..

> nor as efficient as something like NeWS was.
Sure, but I don't think that it's possible to keep current toolkits and have them implement something like NeWS or Fresco/Berlin.

But current toolkits already support X11 so if there was a Wayland2.0 which was (Wayland1.0 + the useful parts of X11 to have network (WAN) access), then it should be possible to have the toolkits use this..

Wayland - Beyond X (The H)

Posted Feb 16, 2012 13:41 UTC (Thu) by farnz (subscriber, #17727) [Link] (2 responses)

XRender is rather limited in functionality, as compared to OpenGL (only considering the 2D rendering case here). As a result, you end up rendering parts of the toolkit as bitmaps (which you ship over the wire as-is when running networks), when a better API like OpenGL lets you offload work to hardware. Toolkit authors want to produce prettier toolkits than XRender can support (see Clutter for an example of a toolkit that makes use of OpenGL features that just can't be efficiently rendered via XRender).

If they're happy with X11's protocol, they're unlikely to go to the effort of porting to Wayland - why bother coding a completely new backend, when your existing X11 backend will already work with Wayland? Equally, why limit yourself to systems running Wayland if you can work with any systems that run X11?

Note that Wayland is still a success if all it does is act as the construction kit for X11 systems that integrate window management, input handling, and compositing. It doesn't need all the clients to talk native Wayland to have succeeded.

Wayland - Beyond X (The H)

Posted Feb 16, 2012 17:18 UTC (Thu) by renox (guest, #23785) [Link] (1 responses)

> XRender is rather limited in functionality, as compared to OpenGL
Yes but:
1) OpenGL is a moving target
2) You cannot simply send OpenGL commands from the client to the server otherwise any command to allocate something would create a round trip.
So you'd have to have a smart proxy on the client, goto (1).
3) With some luck, OpenGL commands to read pixels from buffers would not be an issue as they're already slow, so they're unlikely to be used.

> If they're happy with X11's protocol, they're unlikely to go to the effort of porting to Wayland

I don't understand your remark: there is work to port GTK and Qt to Wayland, clearly some developers want the faster local rendering that Wayland should provide..

The question is about the remote rendering backend: should the X11 backend be kept forever? Can it be improved for WAN access(*)?
Or is-it possible to replace it with something even better for WAN on top of Wayland?

*:After all, in the long run (if Wayland is ported to other Unix), this should be its main purpose.

Wayland - Beyond X (The H)

Posted Feb 16, 2012 18:42 UTC (Thu) by farnz (subscriber, #17727) [Link]

While OpenGL is a moving target, it is a standard, and what's on offer is extended, not reduced. You can choose a useful subset, available on all interesting platforms, and just use that, while still getting more than XRender offers you. Given that people are moving toolkits to OpenGL, there's clearly a demand for more than XRender offers from the perspective of toolkit authors.

Explaining my remark in more depth; nothing in the Wayland world stops people sticking to X11 indefinitely. If Wayland + XRender (or Wayland + OpenGL 1.4 - the upper limit of indirect GLX) is Good Enough, toolkit authors are unlikely to expend the effort to port to Wayland for no net gain. Given that the only gain from Wayland is to permit you to drop legacy code related to the network-transparent subset of X11, the only sensible reason for toolkits to port over and stop caring about X11 is that they want to do things that they can't do network-transparently in X, either.

Therefore, should toolkits stop bothering to maintain a network-transparent X11 backend, it will be because they're doing things that they couldn't do in network-transparent X11. The benefit of Wayland here is that they have to explicitly decide that "not network transparent" is acceptable - they can't simply be an X11 application that does not work over the network due to DRI2 dependencies.

Thus, the toolkits have to make a decision about remote rendering; is it an irrelevance to their users? Is X11 good enough (in which case they will maintain an X11 backend)? Is there a better way (in which case they will implement it, and possibly kill off Wayland in the process)?

Wayland - Beyond X (The H)

Posted Feb 15, 2012 15:55 UTC (Wed) by nix (subscriber, #2304) [Link]

When you think about it, this is mostly what we have today. Most of what core X does (drawing lines, arcs, rendering fonts and the like) is stuff which would these days go into a toolkit. Obviously the point at which this semantic information is available is the point at which network-transparency should be implemented. If you implement it by throwing huge bitmaps around -- whether in a Wayland compositor, or in present-day X handling current Pango-and-Gtk apps over a network -- you're going to get lousy performance.

Wayland - Beyond X (The H)

Posted Feb 15, 2012 19:10 UTC (Wed) by jedidiah (guest, #20319) [Link] (1 responses)

I suppose you could consider Wayland as a way to rebuild Xorg from the ground up but that's not the way it's being presented. It primarily seems to be a rallying point for X haters. The point really seems to be to abandon X entirely. Lip service given to features missing in Wayland just seem to be an attempt to deflect justifiable criticism.

Wayland - Beyond X (The H)

Posted Feb 17, 2012 9:57 UTC (Fri) by alankila (guest, #47141) [Link]

The ultimate problem here is that getting good local and remote network performance simultaneously is very hard. And we currently sacrifice great deal of local performance through having a middle-management layer such as X, or protocols like GLX, or AIGLX, or whatever.

The idea that clients have direct access to GPU and can render directly with it, and everything else is told to get out of the way will allow superb local performance, but at the cost of removing X-style network transparency because there is no component other than the client and the GPU that even knows what is being rendered. So the ways to get network transparency are:

1) request GPU for textures, try to compress them and send them;
2) use toolkit which actually has the semantic knowledge of what is being displayed and can use some intelligent interface to represent the gui data.

Hence the perfectly reasonable request that toolkits should do the network transparency, because they have the semantic information to actually do a good job.