mask current unreliable tests in bionic and cosmic
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
systemd (Ubuntu) |
Fix Released
|
Undecided
|
Unassigned | ||
Bionic |
New
|
Undecided
|
Unassigned | ||
Cosmic |
New
|
Undecided
|
Unassigned |
Bug Description
Hi,
a recent SRU made me check the success rate of systemd autopkgtests.
It looks grim atm:
None of the fails look related to the actual uploads BTW
Bionic:
last 7 ppc64 - http://
last 7 s390x - http://
At least those with "last 7" of them is exactly since systemd/
last 26 amd64 - http://
last 12 i386 - http://
If anything those fail since the last kernel update, but that isn't sure.
last 80 armhf - http://
Here always "root-unittests" fails, maybe not container compatible?
Cosmic:
last ~40 ppc64 - http://
In addition to the "upstream" fails seen in Bionic there are some gdm induced errors which we just fixed in https:/
last 4 s390x - http://
last 16 on amd64 - http://
last 8 arm64 - http://
last 4 i386 - http://
The tests that fail most of the time are "boot-smoke" and upstream"
In general the test "upstream" fails most (amd64/
Maybe that one should be (partially) disabled on the next upload as the rest is at least semi reliable (doable with retries).
The same applies for test "root-unittests" on armhf, please just skip that one and retain the rest.
So maybe on the next systemd update those could be addressed or at least improved, but for now this is fake-blocking way too many SRUs - to lets add an all arch badtest for now.
TL;DR on changes that I'd request for next systemd upload:
- Bionic skip "upstream" test (unless you have the magic dust to make it good again)
- Bionic skip root-unittest on armhf
- Cosmic skip "upstream" test
- Cosmic also implement gdm3 fix we just had in Disco
- Cosmic skip boot-smoke test on arm64 & amd64
I'm leaving this bug open to consider it on the next systemd upload as suggested.
For now I see no other way than to propose an MP to force-badtest.
Changed in systemd (Ubuntu): | |
status: | New → Fix Released |
(for now) Britney hints are supplied to unblock several SRUs that block on these tests while not causing it.
This bug will stay open to (hopefully) track and address it for real on the next systemd upload.
=> https:/ /code.launchpad .net/~paelzer/ britney/ hints-ubuntu- bionic- sru-systemd/ +merge/ 359987 /code.launchpad .net/~paelzer/ britney/ hints-ubuntu- cosmic- sru-systemd/ +merge/ 359986
=> https:/