check for common issues on Failed tests

Bug #1663497 reported by Christian Ehrhardt 
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
autopkgtest (Ubuntu)
New
Undecided
Unassigned

Bug Description

Hi,
as all packagers every now and then I debug cases that unexpectedly fail dep8 tests.
All of us especially love issues that at first seems unreproducible locally but always (or even only sometimes) trigger on Launchpad.

In at least two occasions I had OOM killers kick in and I wondered how much easier I could have found that if there would be something in the logs about that.

So I wanted to ask if any of the following would appear reasonable to implement:
1) In the FAIL case making a few default checks and put it to the logs (can be a dmesg | grep -i oom to start, but I'm sure over time people would come up with more).

2) dmesg as default test artifact for every Test; run after the test is over (maybe only on FAIL) and added to artifacts.

3) Now a bit hardcore, but maybe the de-facto super-debug-artifact for FAIL cases could be to (in that case) install and run sosreport and add the generated tgz as artifact.

summary: - auto-check for oom on Fails
+ check for common issues on Failed tests
Revision history for this message
Martin Pitt (pitti) wrote :

Bug 1288529 is a more generic approach to this, which doesn't hardcode knowledge about journals, sosreports, etc. into autopkgtest itself, and makes it configurable through the CI system.

Please note that when designing this you make sure that you don't collect large logs for every failure. We run hundreds or sometimes thousands of tests every day, and their artifacts all need to be stored. So an sosreport-style tarball which is in the magnitude of a megabyte (or more?) seems prohibitively expensive to collect always.

Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

Yes Bug 1288529 is an evolved version of my 1) and I like it.
The extensibility is good as well and once implemented we could discuss what would be a reasonable --teardown when running on autopkgtest infrastructure.

I agree that collecting huge logs has to be default off, but would be great if configurable.
--teardown would provide that.

Maybe once done another parameter to the retest URL, &teardown="you-command"

Closing mine and subscribing to the dup!

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.