[testsfail] ClientLatency.triple_buffered_client_has_less_than_two_frames_latency

We have been talking about separating performance tests out of the acceptance tests.

This is a recurring problem for time-sensitive tests. Our CI machines are sometimes too slow to pass them.

03:18:00 12: [ debug ] 98 frames sampled, averaging 2.7 frames latency
03:18:00 12: [ debug ] 0: 1 2 2 2 2 2 2 2 2 2
03:18:00 12: [ debug ] 10: 2 2 2 2 2 2 2 2 2 2
03:18:00 12: [ debug ] 20: 2 2 2 2 2 2 2 3 3 3
03:18:00 12: [ debug ] 30: 3 3 3 3 3 3 3 3 3 3
03:18:00 12: [ debug ] 40: 3 3 3 3 3 3 3 3 3 3
03:18:00 12: [ debug ] 50: 3 3 3 3 3 3 3 3 3 3
03:18:00 12: [ debug ] 60: 3 3 3 3 3 3 3 3 3 3
03:18:00 12: [ debug ] 70: 3 3 3 3 3 3 3 3 3 3
03:18:00 12: [ debug ] 80: 3 3 3 3 3 3 3 3 3 3
03:18:00 12: [ debug ] 90: 3 3 3 3 3 3 3 3
03:18:00 12: /��BUILDDIR��/mir-0.23.0+vivid1061bzr3497/tests/acceptance-tests/test_latency.cpp:296: Failure
03:18:00 12: Value of: observed_latency
03:18:00 12: Expected: is < 2.4
03:18:00 12: Actual: 2.71429 (of type float)
03:18:00 12: [2016-05-09 03:18:00.781802] mirserver: Stopping
03:18:00 12: [ FAILED ] ClientLatency.triple_buffered_client_has_less_than_two_frames_latency (1762 ms)

https://mir-jenkins.ubuntu.com/job/build-2-binpkg-mir/arch=amd64,compiler=clang,platform=mesa,release=vivid+overlay/1034/consoleFull

And now:

06:32:57 12: [ debug ] 98 frames sampled, averaging 2.7 frames latency
06:32:57 12: [ debug ] 0: 1 2 2 2 2 2 2 2 2 2
06:32:57 12: [ debug ] 10: 2 2 2 2 2 2 2 2 2 2
06:32:57 12: [ debug ] 20: 2 2 2 2 2 2 2 2 2 2
06:32:57 12: [ debug ] 30: 2 2 2 3 3 3 3 3 3 3
06:32:57 12: [ debug ] 40: 3 3 3 3 3 3 3 3 3 3
06:32:57 12: [ debug ] 50: 3 3 3 3 3 3 3 3 3 3
06:32:57 12: [ debug ] 60: 3 3 3 3 3 3 3 3 3 3
06:32:57 12: [ debug ] 70: 3 3 3 3 3 3 3 3 3 3
06:32:57 12: [ debug ] 80: 3 3 3 3 3 3 3 3 3 3
06:32:57 12: [ debug ] 90: 3 3 3 3 3 3 3 3
06:32:57 12: /��BUILDDIR��/mir-0.23.0+vivid1074bzr3510/tests/acceptance-tests/test_latency.cpp:296: Failure
06:32:57 12: Value of: observed_latency
06:32:57 12: Expected: is < 2.4
06:32:57 12: Actual: 2.65306 (of type float)
06:32:57 12: [2016-05-10 06:32:57.700788] mirserver: Stopping
06:32:57 12: [ FAILED ] ClientLatency.triple_buffered_client_has_less_than_two_frames_latency (1810 ms)

https://mir-jenkins.ubuntu.com/job/build-2-binpkg-mir/arch=amd64,compiler=clang,platform=mesa,release=vivid+overlay/1046/consoleFull

And another one: https://mir-jenkins.ubuntu.com/job/build-2-binpkg-mir/arch=amd64,compiler=clang,platform=mesa,release=vivid+overlay/1092/consoleFull

^^^ says:

16:05:00 12: [ debug ] 98 frames sampled, averaging 2.6 frames latency
16:05:00 12: [ debug ] 0: 1 2 2 2 2 2 2 2 2 2
16:05:00 12: [ debug ] 10: 2 2 2 2 2 2 2 2 2 2
16:05:00 12: [ debug ] 20: 2 2 2 2 2 2 2 2 2 3
16:05:00 12: [ debug ] 30: 3 3 2 2 2 2 2 2 3 3
16:05:00 12: [ debug ] 40: 3 3 3 3 3 3 3 3 3 3
16:05:00 12: [ debug ] 50: 3 3 3 3 3 3 3 3 3 3
16:05:00 12: [ debug ] 60: 3 3 3 3 3 3 3 3 3 3
16:05:00 12: [ debug ] 70: 3 3 3 3 3 3 3 3 3 3
16:05:00 12: [ debug ] 80: 3 3 3 3 3 3 3 3 3 3
16:05:00 12: [ debug ] 90: 3 3 3 3 3 3 3 3
16:05:00 12: /��BUILDDIR��/mir-0.23.0+vivid1121bzr3512/tests/acceptance-tests/test_latency.cpp:296: Failure
16:05:00 12: Value of: observed_latency
16:05:00 12: Expected: is < 2.4
16:05:00 12: Actual: 2.63265 (of type float)
16:05:00 12: [2016-05-19 16:05:00.057651] mirserver: Stopping
16:05:00 12: [ FAILED ] ClientLatency.triple_buffered_client_has_less_than_two_frames_latency (1821 ms)

It's almost suspicious we're missing the mark by 0.2 frames each time. Such a small amount...

https://mir-jenkins.ubuntu.com/job/build-2-binpkg-mir/arch=amd64,compiler=gcc,platform=mesa,release=xenial/1095/consoleFull

17:04:09 12: [ RUN ] ClientLatency.triple_buffered_client_has_less_than_two_frames_latency
17:04:09 12: [2016-05-19 17:04:09.111980] mirserver: Starting
17:04:09 12: [2016-05-19 17:04:09.112247] mirserver: Selected driver: dummy (version 0.23.0)
17:04:09 12: [2016-05-19 17:04:09.117036] mirserver: Using software cursor
17:04:09 12: [2016-05-19 17:04:09.117464] mirserver: Initial display configuration:
17:04:09 12: [2016-05-19 17:04:09.121744] mirserver: Selected input driver: mir:stub-input (version: 0.23.0)
17:04:09 12: [2016-05-19 17:04:09.121938] mirserver: Mir version 0.23.0
17:04:10 12: [ debug ] 99 frames sampled, averaging 2.5 frames latency
17:04:10 12: [ debug ] 0: 1 2 2 2 2 2 2 2 2 2
17:04:10 12: [ debug ] 10: 2 2 2 2 2 2 2 2 2 2
17:04:10 12: [ debug ] 20: 2 2 2 2 2 2 2 2 2 2
17:04:10 12: [ debug ] 30: 2 2 2 2 2 2 2 2 2 2
17:04:10 12: [ debug ] 40: 2 2 2 2 2 3 3 3 3 3
17:04:10 12: [ debug ] 50: 3 3 3 3 3 3 3 3 3 3
17:04:10 12: [ debug ] 60: 3 3 3 3 3 3 3 3 3 3
17:04:10 12: [ debug ] 70: 3 3 3 3 3 3 3 3 3 3
17:04:10 12: [ debug ] 80: 3 3 3 3 3 3 3 3 3 3
17:04:10 12: [ debug ] 90: 3 3 3 3 2 2 2 2 2
17:04:10 12: /��BUILDDIR��/mir-0.23.0+xenial1124bzr3512/tests/acceptance-tests/test_latency.cpp:296: Failure
17:04:10 12: Value of: observed_latency
17:04:10 12: Expected: is < 2.4
17:04:10 12: Actual: 2.48485 (of type float)
17:04:10 12: [2016-05-19 17:04:10.860621] mirserver: Stopping
17:04:10 12: [ FAILED ] ClientLatency.triple_buffered_client_has_less_than_two_frames_latency (1764 ms)

Fix committed into lp:mir at revision None, scheduled for release in mir, milestone 0.24.0

This bug was fixed in the package mir - 0.24.0+16.10.20160815.3-0ubuntu1

---------------
mir (0.24.0+16.10.20160815.3-0ubuntu1) yakkety; urgency=medium

  * New upstream release 0.24.0 (https://launchpad.net/mir/+milestone/0.24.0)
    - ABI summary:
      . mirclient ABI unchanged at 9
      . mirserver ABI bumped to 41
      . mircommon ABI unchanged at 6
      . mirplatform ABI bumped to 12
      . mirprotobuf ABI unchanged at 3
      . mirplatformgraphics ABI bumped to 10
      . mirclientplatform ABI unchanged at 5
      . mirinputplatform ABI unchanged at 5
    - Enhancements:
      . Added a new tool 'mirvanity' which can use a high speed video camera
        to measure client-to-screen latency.
      . Added (build-time) support for desktop full OpenGL servers (disabled
        by default in favor of GLESv2).
      . Introduced new buffer semantics (NBS) and enabled it by default.
      . Avoided using libmirserver in mir_demo_standalone_render_overlays,
        making bringup traces less cluttered.
      . Removed TilingWindowManager from example servers.
      . Added two new mir_*_performance_test tools that are run as part of
        'mir_performance_tests'.
      . Added pointer confinement to the client API.
      . Added new platform supporting software EGLStreams.
      . Added an input platform that the nested server can use.
      . Allow configuration of the application_not_responding_detector
      . Handle server-side keymapping using XKBMapper.
      . Remove the offscreen display.
      . Add callback option to notify shells that the server is about to stop.
      . Add logging for ANativeWindow events on Android.
    - Bugs fixed:
      . usage of std:call_once in mirclient causes TLS collisions with some
        android devices (LP: #1599867)
      . AltGr not working on external keyboards (LP: #1565236)
      . [regression] unity8 fails to start when built with 0.24 series (lp:mir)
        (LP: #1597717)
      . CI failure in MirSurfaceVisibilityEvent.exposed_received_when_surface_
        raised_over_occluding_surface [called twice - over-saturated and active]
        (LP: #1556045)
      . package-built mir_demo_server does not start on device (LP: #1577357)
      . During surface creation, first stream in spec becomes default stream.
        (LP: #1577967)
      . mouse is getting stuck on a phantom edge (LP: #1580774)
      . [testsfail] in MirSurfaceVisibilityEvent.exposed_received_when_surface_
        raised_over_occluding_surface [never called] (LP: #1581385)
      . [enhancement] Add support for full OpenGL compositing (LP: #1420581)
      . [testsfail] ServerSignal.terminate_handler_is_called_for_SIGINT|SIGTERM
        (LP: #1570353)
      . [testsfail] ClientLatency.triple_buffered_client_has_less_than_two_
        frames_latency (LP: #1576690)
      . [testsfail] NestedServer.when_monitor_plugged_in_client_is_notified_of_
        new_display_configuration (LP: #1576760)
      . mir server crashed in what(): drmModeMoveCursor failed (returned -13)
        (LP: #1579630)
      . mirtest-dev is hard to use as the objects used are compiled with LTO
        (LP: #1583536)
      . [testsfail] ClientLogging.repo...