[regression] Input focus delay after switching app back into focus since OTA5

Status changed to 'Confirmed' because the bug affects multiple users.

This is an input/focus bug primarily.

Not related to any latency improvements I'm working on. The first of those won't arrive until Mir 0.15.

Confirmed. And surprisingly confirmed the problem is in Mir itself.

I can reproduce the same focus delay problem with pure Mir demos, run mir_proving_server and switching apps with a four-finger swipe. The same delay occurs in that any gesture started within about one second of the focus switch is ignored (until you lift your finger and try again).

Bisected. Actually, trisected. There were three separate regressions to input and focus (one of them fixed), and we're now seeing the cumulative effect of the other two.

First regression: r2590 slightly broke focus changing such that the focus change is sometimes withheld and delayed until your touch gesture in the old window finishes. It doesn't matter how long your gesture takes, since r2590 Mir would delay the focus switch till the gesture is completed (fingers raised).

Second regression: r2652 broke touch input completely. It did not work at all until fixed in r2656 (LP: #1464174).

Third regression: Some combination of r2652-r2656 created the behaviour we see today. That is the focus change takes effect immediately but touch events are ignored for about the first second after the focus change. I can't tell which revision caused the exact problem because the second regression remained unresolved during this time so there was no working touch support to test with.

The exact offending revisions to investigate are:

------------------------------------------------------------
revno: 2590 [merge]
author: Robert Carr <email address hidden>, <email address hidden>
committer: Tarmac
branch nick: development-branch
timestamp: Thu 2015-05-21 02:17:18 +0000
message:
  Unify MirMotionTooltype and MirTouchTooltype.

  Approved by PS Jenkins bot, Andreas Pokorny, Alberto Aguirre, Kevin DuBois, Al
exandros Frantzis.
------------------------------------------------------------
------------------------------------------------------------
revno: 2652 [merge]
author: Robert Carr <email address hidden>, <email address hidden>
committer: Tarmac
branch nick: development-branch
timestamp: Wed 2015-06-10 23:37:28 +0000
message:
  Replace android input dispatcher with a slimmed down rewrite suitable to Mir's
 requirements. Fixes: https://bugs.launchpad.net/bugs/1419048.

  Approved by PS Jenkins bot, Alexandros Frantzis, Andreas Pokorny, Chris Halse
Rogers.
------------------------------------------------------------
------------------------------------------------------------
revno: 2656 [merge]
author: <email address hidden>
committer: Tarmac
branch nick: development-branch
timestamp: Fri 2015-06-12 05:34:31 +0000
message:
  Fix key repeat dispatcher (touch events not working at all)
  (LP: #1464174). Fixes: https://bugs.launchpad.net/bugs/1464174.

  Approved by Daniel van Vugt, Alberto Aguirre, PS Jenkins bot.
------------------------------------------------------------

Ironically, Robert has departed the company as of 3 days ago :/

marking as high.
also, @duflu - is it possible we can add an integration test in place to catch future potential regression for this use case on ci ?

I disagree with High. Not only is the bug so minor that it went unnoticed during the release of 0.14, but even now it took quite some effort to discover. That's not High (which to me would imply it's a potential release blocker). Not a major bug, IMHO.

Spoke too soon. It seems the bug reproducible using lp:mir is even worse than the original bug seen with Unity. Using lp:mir you see all the bugs...

  1. Focus switch does not occur immediately. You need to start and finish a new gesture before it takes effect (e.g. touch the screen).
  2. When the new window eventually comes on top, you still can't interact with it. Not until you do yet another gesture (e.g. touch the screen and release). After that, you can finally interact with the new window.

So that's starting to sound like High severity. Although it doesn't appear to be as bad on arale when running Unity (yet).

Also made a minor mistake. Above where I mentioned r2590, that should be r2591 where the first regression occurred. r2590 was actually the last revision where everything worked properly...

------------------------------------------------------------
revno: 2591 [merge]
author: <email address hidden>, Robert Carr <email address hidden>
committer: Tarmac
branch nick: development-branch
timestamp: Thu 2015-05-21 04:17:33 +0000
message:
  Prefer using the event builders.

  Approved by Alberto Aguirre, Chris Halse Rogers, PS Jenkins bot.
------------------------------------------------------------

As we have at least two separate clear points of regression I have moved the first one into a new bug 1481570.

OK, the first bug 1481570 has a fix up for review. That's faulty end-of-multifinger gesture detection. Probably doesn't apply to Unity, just the Mir demo server..

The second bug (the Mir part of this one) seems to be a simple matter of faulty end-of-gesture detection introduced in r2652. The exact problem seems to be the new SurfaceInputDispatcher::dispatch_touch gesture_owner logic that was introduced. It assumes there's no shell that might consume your touch events in handling app-switching gestures. If the shell does correctly consume those touch events that triggered the app switch then the gesture_owner logic introduced in r2652 doesn't work and it keeps delivering touches to the old window (which is now hidden) instead of the new one.

So that needs fixing in Mir. But it doesn't mean there aren't additional timing bugs in other components after that... :/

I need to point out that unity8 is not using much of Mir's input dispatching & focus logic, so please be certain the reported issue is actually resolved with these mir fixes.

I'm somewhat confident the Mir fix does _not_ fix this bug entirely. Just fixing all the related bugs on the Mir side so everything that's left is not Mir :)

Tested the Mir fix on the full phone. The bug is not entirely solved, so more work is indeed required looking to qtmir/unity8.

It looks like QtMir/Unity8 is making the same mistake as Mir did. See the Mir fix for a hint of what to fix.

It also feels like the Unity8 window switcher is getting in the way. Particularly when switching into the web browser, it takes half a second or so to finish the app switch after the animation apparently completes. You can notice this just slightly as a flicker of shadow (from the app switcher) appears on the right briefly, and sometimes the app surface goes from slightly transformed (animation didn't end at the right spot) to suddenly completely flat in that time. So it feels and looks like the Unity8 app switcher is not finished doing it's thing for about half a second after the animation apparently completes.

Fix committed into lp:mir at revision None, scheduled for release in mir, milestone 0.16.0

Still not yet sure where the remaining delay is. Seems most likely in Unity8 as that's where the spread animation happens.

Certainly the bug is fixed on the Mir side, but it's a very weird coincidence that I found such similar regressions in Mir in the same release, even if they're not the main problem here.

This bug was fixed in the package mir - 0.15.0+15.10.20150818-0ubuntu1

---------------
mir (0.15.0+15.10.20150818-0ubuntu1) wily; urgency=medium

  [ Daniel van Vugt ]
  * New upstream release 0.15.0 (https://launchpad.net/mir/+milestone/0.15.0)
    - ABI summary: Only servers and graphics drivers need rebuilding;
      . Mirclient ABI unchanged at 9
      . Mirserver ABI bumped to 33
      . Mircommon ABI unchanged at 5
      . Mirplatform ABI bumped to 9
    - Enhancements:
      . Add support for Mir-on-X11.
      . Latency reduction optimizations (around ~15ms reduction in total):
        Reduced input event resampling latency by 5ms. Reduced output latency
        (in system compositors) by around 10ms with the introduction of
        "predictive bypass". And we're not finished; future Mir releases
        should reduce latency further.
      . Introduced a python3-based Mir performance framework.
      . Lots of preparation for an architectural overhaul of buffer swapping,
        required in the least to support future optimizations like nested
        bypass.
      . Added a new cursor: crosshair
      . Added support for 15/16-bit client pixel formats ("high colour").
      . Added a new client function to make picking the right pixel format
        for a given EGLConfig super simple: mir_connection_get_egl_pixel_format
      . Added application-not-responding detection
      . Added client API for specifying input region shape.
      . Fixed the remaining threading flaws identified by ThreadSanitizer and
        turned it on permanently for all continuous integration in future.
      . Added support for relative pointer motion events (e.g. for gaming).
    - Bug fixes:
      . Fix focus issues breaking autopilot tests entering text (LP: #1468029)
      . Fix mir tests failure on armhf with GCC5 (LP: #1478213)
      . mir_buffer_stream_swap_buffers_sync can hang constraints (LP: #1479899)
      . Loading libmirclient.so twice leads to a segfault in libmirprotobuf.so
        (LP: #1391976)
      . Visible corruption in SDL apps (LP: #1460149)
      . MultiThreadedCompositor::destroy_compositing_threads hangs/deadlocks on
        shutdown or display reconfiguration (LP: #1471909)
      . ctest/"make test" reports 100% tests pass even when some fail.
        (LP: #1472911)
      . Mir server crashed - GLib-CRITICAL **: g_source_get_context: assertion
       'source->context != NULL || !SOURCE_DESTROYED (source)' failed
        (LP: #1473869)
      . USC crash on multimonitor unplug [std::exception::what: error during
        hwc prepare()] (LP: #1474891)
      . [regression] Input focus delay after switching app back into focus
        (LP: #1480654)
      . GLibMainLoopTest.propagates_exception_from_server_action fails with
        GCC 5 in armhf (LP: #1482274)
      . [enhancement] Mir lacks relative mouse support (LP: #1276322)
      . ShmBuffer ignores pixel_format (LP: #1424909)
      . Fullscreen bypassed clients stutter with double buffers when other
        clients are running (LP: #1447896)
      . [regression] Demo servers crash on start-up if MIR_ENABLE_TESTS=OFF
        (LP: #1439078)
      . [regression] The software curs...

note that this behaviour is even more noticeable on turbo, seems the Mir slowness is out of the way there, it takes between two and three seconds for an app to accept input ... of which one second is a little "hang" of the animation when flipping it into focus before it sits straight on the screen, this seems like some blocking code in unity8.

Last time I tested this bug (and the Mir fix) the only remaining problem for me was that one second tick of the Unity8 app switch animation completing and then realigning the surface to fit the screen. Even that delay doesn't happen reliably on demand for me...

I have just retested OTA-10.1 on krillin and can reproduce a second or so of delay, but only after selecting an app like Calculator from the full spread.

Annoyingly the OSK makes things worse in a different way -- it's so slow to show and hide that it seems to be masking this bug. So you need to test apps that don't need the OSK but do expect touches.

Status changed to 'Confirmed' because the bug affects multiple users.

FWIW, I can only reproduce this if either the resumed or the suspended app is a webapp (or the browser). Perhaps this is related to the fact that oxide runs in separate processes and we're sending sigstop/sigcont to a chain of processes?

so shall we refine this bug or spawn a new one? sounds like this is now beyond the original regression that was solved

well, it was never actually solved completely ... (as you can see by daniels comment above)
i suspect there are still multipple bugs ...

there is:

a) the spread animation getting stuck a few pixels before the window sits completely straight on the screen for a split second when you switch apps (doesnt happen every time but well noticeable throughout the day).

b) input on re-focus (doesnt matter if with or without spread involved) for all apps using a webview acts delayed (try my G+ app or any other that uses the bottom navigation, bring the app in focus and try to swipe the navigation menu up ... you will notice it takes 3-4 swipes til there is any reaction))

c) general input delay when unlocking ... it usually takes multiple swipes after unlocking to get a reaction (for spread, launcher or app interaction)

i think a) and c) are still this bug ... b) should perhaps be an oxide or unity8 one assuming mzanetti is right above.

I can confirm that (a) (b) and (c) in Oliver's comment [1] affects me.

I will add another:

d) unresponsive to power button press to wake up device. I sometimes need to press a dozen or more times before the display will respond and I'm presented with a login screen, or sometimes with the restart/cancel, etc option menu.

About:
Nexus 7 (2013)
Ubuntu 15.04 (r467)

Cheers,
Randall

[1] https://bugs.launchpad.net/ubuntu/+source/unity8/+bug/1480654/comments/26

I'm getting (b) (https://bugs.launchpad.net/ubuntu/+source/unity8/+bug/1480654/comments/26) as well on the

Nexus 4 (Mako)

particularly in the browser, which also leads to the fast scrolling bug:
https://bugs.launchpad.net/ubuntu/+source/webbrowser-app/+bug/1584965