Some unit tests fail when run under an sbuild chroot

Relevant documentation: https://pkg-perl.alioth.debian.org/debhelper.html#running_tests_needing_writable_home

The linked branch fixes the failures for tst_FaviconFetcherTests and tst_QmlTests, but the failure for tst_SingleInstanceManagerTests remains.

I reproduced the tst_SingleInstanceManagerTests failure locally in an schroot, here is the XML output of the test run:

<?xml version="1.0" encoding="UTF-8" ?>
<testsuite errors="2" failures="2" tests="4" name="SingleInstanceManagerTests">
  <properties>
    <property value="5.5.1" name="QTestVersion"/>
    <property value="5.5.1" name="QtVersion"/>
    <property value="Qt 5.5.1 (x86_64&#x002D;little_endian&#x002D;lp64 shared (dynamic) release build; by GCC 5.3.1 20160330)" name="QtBuild"/>
  </properties>
  <testcase result="pass" name="initTestCase"/>
  <testcase result="fail" name="test_cannot_run_twice_same_instance">
    <!-- message="Failed to launch single instance: &quot;QLocalServer::listen: Name error&quot;" type="system" -->
    <failure message="&apos;singleton&#x002D;&gt;run(QStringList())&apos; returned FALSE. ()" result="fail"/>
  </testcase>
  <testcase result="fail" name="test_arguments_passed_to_already_running_instance">
    <!-- message="Failed to launch single instance: &quot;QLocalServer::listen: Name error&quot;" type="system" -->
    <failure message="&apos;singleton&#x002D;&gt;run(QStringList())&apos; returned FALSE. ()" result="fail"/>
  </testcase>
  <testcase result="pass" name="cleanupTestCase"/>
  <system-err>
<![CDATA[Failed to launch single instance: "QLocalServer::listen: Name error"]]>
<![CDATA[Failed to launch single instance: "QLocalServer::listen: Name error"]]>
  </system-err>
</testsuite>

When running the tests in an sbuild chroot, I see the following:

    HOME=/<<BUILDDIR>>/webbrowser-app-0.23+16.04.20160405/debian/build dh_auto_test

If I disable log filtering in sbuild configuration, it looks like this:

    HOME=/build/webbrowser-app-4ccXnp/webbrowser-app-0.23+16.04.20160405/debian/build dh_auto_test

I’ve added some logging to single-instance-manager.cpp, and the test results look like this:

PASS : SingleInstanceManagerTests::initTestCase()
QDEBUG : SingleInstanceManagerTests::test_cannot_run_twice_same_instance() bool SingleInstanceManager::run(const QStringList&) "/build/webbrowser-app-4ccXnp/webbrowser-app-0.23+16.04.20160405/debian/build/.qttest/share/tst_SingleInstanceManagerTests/SingletonSocket"
QSYSTEM: SingleInstanceManagerTests::test_cannot_run_twice_same_instance() Failed to launch single instance: "QLocalServer::listen: Name error"
FAIL! : SingleInstanceManagerTests::test_cannot_run_twice_same_instance() 'singleton->run(QStringList())' returned FALSE. ()
   Loc: [/build/webbrowser-app-4ccXnp/webbrowser-app-0.23+16.04.20160405/tests/unittests/single-instance-manager/tst_SingleInstanceManagerTests.cpp(53)]
QDEBUG : SingleInstanceManagerTests::test_arguments_passed_to_already_running_instance() bool SingleInstanceManager::run(const QStringList&) "/build/webbrowser-app-4ccXnp/webbrowser-app-0.23+16.04.20160405/debian/build/.qttest/share/tst_SingleInstanceManagerTests/SingletonSocket"
QSYSTEM: SingleInstanceManagerTests::test_arguments_passed_to_already_running_instance() Failed to launch single instance: "QLocalServer::listen: Name error"
FAIL! : SingleInstanceManagerTests::test_arguments_passed_to_already_running_instance() 'singleton->run(QStringList())' returned FALSE. ()
   Loc: [/build/webbrowser-app-4ccXnp/webbrowser-app-0.23+16.04.20160405/tests/unittests/single-instance-manager/tst_SingleInstanceManagerTests.cpp(60)]
PASS : SingleInstanceManagerTests::cleanupTestCase()

So the single instance manager fails to create a local server listening on /build/webbrowser-app-4ccXnp/webbrowser-app-0.23+16.04.20160405/debian/build/.qttest/share/tst_SingleInstanceManagerTests/SingletonSocket

The failure seems to be due to the pathname of the domain socket being too long ( > 108 chars, see man page for 'unix').
I wonder if setting BUILDHOME to something like /tmp/home would work (assuming that /tmp always exists).

This bug was fixed in the package webbrowser-app - 0.23+16.04.20160408.1-0ubuntu1

---------------
webbrowser-app (0.23+16.04.20160408.1-0ubuntu1) xenial; urgency=medium

  [ CI Train Bot ]
  * Resync trunk.

  [ Loïc Molinari ]
  * Made AddressBar height scalable with regards to the grid units
    system.

  [ Olivier Tilloy ]
  * Add dep8 tests and instructions to run them in qemu or on a phone.
    Original work by Leo Arias and Vincent Ladeuil. added: debian/tests/
    debian/tests/control debian/tests/touch-session-autopilot
  * Catch ESC key event one level up to ensure that it’s not incorrectly
    bubbled up to the outer component. (LP: #1557016)
  * Customize the contents of the media permission dialog to avoid
    truncated text. (LP: #1554220)
  * Do not write the session to a temporary file when no target file is
    defined.
  * Fix a failing unit test with Qt 5.6. (LP: #1565507)
  * Fix broken webapp container autopilot tests. (LP: #1557019)
  * Fix issues with item selection in the downloads page: do not allow
    selecting multiple files when only one is expected do not allow
    entering delete mode (with a long press on an item) while in picker
    mode (LP: #1534112, #1561575)
  * Fix unit tests when run under an sbuild chroot. (LP: #1567294)
  * Import QQuickShortcut from Qt 5.5 to properly handle window-level
    keyboard shortcuts. We cannot bump the dependency on Qt to 5.5 as
    the stable overlay PPA for devices currently has Qt 5.4.1. (LP:
    #1542224, #1545802, #1537782)
  * Make the autopilot tests more reliable when dragging the bottom
    edge. (LP: #1560109)
  * Remove workaround for bug #1526940 that was fixed in the latest
    release of the UITK. (LP: #1526940)
  * Rename debian packages to follow new QML module naming convention.
    (LP: #1342031)
  * Simplify the implementation of HistoryViewWide quite a bit, and as a
    side effect fix a unit test failure when run against the staging
    branch of the UITK. (LP: #1567337)

 -- Olivier Tilloy <email address hidden> Fri, 08 Apr 2016 17:07:04 +0000