Unity8 pointer does not stay in sync with Qemu VM tablet input

I think this might be due to us relying on relative (mouse/trackballs/trackpoints) instead of absolute (graphics tablets) input data.

Or, possibly, the lack of hardware cursor for nested servers still?

Yes, anpok was talking about this problem in a meeting yesterday.

The issue is that U8 doesn't yet understand that some devices are absolute and not relative. As such U8 tracking it's own cursor position won't work.

I think in the long run to solve this bug (and to reduce cursor lag, and to eliminate the rendering load of U8 cursor movement) we will need to switch to using native Mir input coordinates as well as the hardware cursor.

AFAIK the only thing stopping us from using Mir's native input coordinates was the inability to implement barriers/edge resistance in places. But I guess we could add an API for that (also discussed in the same meeting yesterday).

I'm not totally convinced that this bug is about relative pointer movement. For example, if I leave the pointer at one corner of the virtual screen, then move it outside and all the way around the edge, then move it back inside the virtual screen at the opposite corner, the virtual pointer moves more or less to the bottom right.

Also, given that the pointers gradually calibrate and move more closely in sync with each other, it feels like there's some "estimate" for the mapping between the size of the graphics tablet and the screen that is improving as you increase the range of coordinates that U8 has sampled.

> AFAIK the only thing stopping us from using Mir's native input coordinates
> was the inability to implement barriers/edge resistance in places.

We had other use-cases where unity8 having control over mouse position was useful. One example coming to mind is phone docked to an external display, we wanted to limit the cursor to the external display only. And we had a few other pie-in-the-sky design ideas which would be much easier if unity8 had cursor control.

I still don't see why Mir can't just let Unity8 (a nested) position the cursor (like it could, if it were a host server)!

And absolute vs relative. If we're doing it wrong, we can fix it.

Invalid for Mir because the enhancement Gerry mentioned is covered in bug 1600220 instead.

If Unity8 used the absolute pointer coordinates that USC gives it then we wouldn't have a problem here. If Unity8 doesn't want to use those all the time then we will need to work out a compromise that makes events from Qemu's mouse/tablet hybrid more obviously absolute (like touches are) so that clients (which U8 is one) know that relative motion from that device is useless.

In Qemu you can move the cursor off the right edge of the window, move it around the Qemu window and make it appear on some other edge. It would be messy at least to try and communicate that using relative motion data. If we really want to do that then sure this is a Mir bug, but I don't think we want to do that.

Useful links:
https://bugs.launchpad.net/mir/+bugs?field.tag=wacom
https://bugs.launchpad.net/mir/+bugs?field.tag=cursor

> We had other use-cases where unity8 having control over mouse position
> was useful. One example coming to mind is phone docked to an external
> display, we wanted to limit the cursor to the external display only. And
> we had a few other pie-in-the-sky design ideas which would be much
> easier if unity8 had cursor control.

Exactly this, btw, is still in play with our multi-monitor designs.

Actually the Mir hack I describe in comment #7 would not work because it assumes the velocity and acceleration of USC and Unity8 cursors is identical.

We'll just need to find a way to ensure Unity8 knows some devices are absolute (which is required for pen tablets in general - see https://bugs.launchpad.net/mir/+bugs?field.tag=wacom).

Looking into QtMir, we're using relative

Looking into QtMir, we're only using the relative mouse motion in our calculation of the cursor position. We need that to implement the push-against-screen-edge behaviour for a mouse.

We've not considered tablet input devices at all. I see Mir's associates a device with each input event, so it possible for us to change behaviour depending on the input device. Overall I don't knom enough to speak intelligently about what we need to do, needs further investigation on our side.

Whatever the solution, we need to be aware that a stylus can jump all over the screen but also move like a normal mouse cursor if it's in close proximity to (even not touching) the screen. The shell has no say in the movement then because the cursor needs to match the real-world stylus position, even when moving like a mouse.

I think that's the kind of device Qemu is emulating but also suspect Qemu's fake tablet is more complicated than that -- a mouse rather than a tablet, which emits absolute coordinates that can suddenly appear from any side of the virtual screen.