apache2 run-test-suite fails frequently in autopkgtest in groovy/armhf

Some of the other test cases appear to randomly fail, in addition to #17:

t/apache/expr.t ..................... ok
# Failed test 11 in t/apache/expr_string.t at line 87
# Failed test 14 in t/apache/expr_string.t at line 101 fail #3
# Failed test 17 in t/apache/expr_string.t at line 101 fail #4
# Failed test 20 in t/apache/expr_string.t at line 101 fail #5
# Failed test 23 in t/apache/expr_string.t at line 101 fail #6
# Failed test 26 in t/apache/expr_string.t at line 101 fail #7
# Failed test 29 in t/apache/expr_string.t at line 101 fail #8
t/apache/expr_string.t ..............
Failed 7/29 subtests

t/apache/expr.t ..................... ok
# Failed test 14 in t/apache/expr_string.t at line 87
# Failed test 17 in t/apache/expr_string.t at line 87 fail #2
# Failed test 20 in t/apache/expr_string.t at line 101 fail #4
# Failed test 23 in t/apache/expr_string.t at line 101 fail #5
# Failed test 26 in t/apache/expr_string.t at line 87 fail #3
t/apache/expr_string.t ..............
Failed 5/29 subtests
t/apache/getfile.t .................. ok

Looking at the code for this test, it feels familiar. I think Andreas and I investigated a similar issue some months ago. I seem to recall we suspected apache was either segfaulting or timing out. I don't recall if that was on armhf but it may have been.

What the test in expr_string.t is doing is to craft .htaccess files with invalid expressions like 'foo' or '}' and then perform GET() requests against the local apache server, pulling the url '/apache/expr/index.html'. It then examines the error.log's info debug lines to check if what got logged is sensible.

Upstream appears to have hit the error before as well:

http://mail-archives.apache.org/mod_mbox/httpd-dev/202001.mbox/%<email address hidden>%3E
http://mail-archives.apache.org/mod_mbox/httpd-dev/202001.mbox/%3C20200113145056.GA12523%40redhat.com%3E
https://travis-ci.org/github/apache/httpd/jobs/633442262

"""
I'm guessing this is some kind of timing issue in the test case, which
tries to read an error_log entry immediately after the request which
generates it. I've added a sleep which will maybe mitigate this -
http://svn.apache.org/r1872705 so let's see how this one goes
"""

Checking with Andreas, he had created a branch with additional verbosity:
https://code.launchpad.net/~ahasenack/ubuntu/+source/apache2/+git/apache2

However, with the debug statements, the issue didn't reproduce.

I guess that fits with upstream's hypothesis that there is a race condition reading the error log.

What upstream tested was adding a little sleep:
http://svn.apache.org/viewvc/httpd/test/framework/trunk/t/apache/expr_string.t?r1=1872705&r2=1872704&pathrev=1872705&view=patch

We can cherry-pick that and see if it resolves it for us as well. The test case is in the debian/ tree, so this shouldn't be too invasive.

This bug was fixed in the package apache2 - 2.4.43-1ubuntu2

---------------
apache2 (2.4.43-1ubuntu2) groovy; urgency=medium

  * d/p/t/apache/expr_string.t: Avoid test suite failure due to timing
    issue reading error log too quickly after request, by adding a sleep.
    (LP: #1890302)

 -- Bryce Harrington <email address hidden> Wed, 05 Aug 2020 12:44:59 -0700