Ticket gets marked as 'Complete' (passed) when the package has a failing test

Bug #1289626 reported by Chris Johnston
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Ubuntu CI Engine
Fix Released
Critical
Vincent Ladeuil
Ubuntu CI Services
Fix Released
Critical
Vincent Ladeuil

Bug Description

I created a failing test in autopkgtest to see what would happen... It turns out, bad things happen... The ticket was marked as completed instead of failed.

I added the following to tests/adt-run:

    def test_chris_changes(self):
        self.assertEqual('1', '2')

autopkgtest-subunit-stream contains:

test: dsc0t-adt-run
error: dsc0t-adt-run [ multipart
Content-Type: text/plain;charset=utf8
error
36
adt-run produced an unexpected result: "dsc0t-adt-run"0
]
test: dsc0t-pyflakes
successful: dsc0t-pyflakes [ multipart
Content-Type: text/plain;charset=utf8
stderr
0
Content-Type: text/plain;charset=utf8
stdout
0
]

17:32:26 fginther | oooh, I wonder if this package has multiple tests and we
17:32:45 fginther | we're only checking the return code of the last one.
17:33:03 fginther | because the pyflakes test passed

Tags: airline

Related branches

Revision history for this message
Andy Doan (doanac) wrote :

Vincent - any ideas?

Changed in ubuntu-ci-services-itself:
importance: Undecided → High
assignee: nobody → Vincent Ladeuil (vila)
Vincent Ladeuil (vila)
Changed in ubuntu-ci-services-itself:
status: New → Confirmed
importance: High → Critical
status: Confirmed → In Progress
Vincent Ladeuil (vila)
Changed in ubuntu-ci-services-itself:
status: In Progress → Fix Committed
Revision history for this message
Chris Johnston (cjohnston) wrote :

MP fails testing. In what way is this fix committed?

Revision history for this message
Vincent Ladeuil (vila) wrote : Re: [Bug 1289626] Re: Ticket gets marked as 'Complete' (passed) when the package has a failing test

>>>>> Chris Johnston <email address hidden> writes:

    > MP fails testing.

Where ?

    > In what way is this fix committed?

There is a fix, it is committed in the proposed branch,
fix... committed... get it ?

The difference being for someone visiting the bug that it's no 'in
progress' anymore, that someone may help test the fix for example.

Now, a more constructive approach would have been for you to:

1) Ensure the tests you see failing were reported to the MP

2) Posting the test failures

Andy Doan (doanac)
Changed in ubuntu-ci-services-itself:
status: Fix Committed → Fix Released
Ursula Junque (ursinha)
Changed in uci-engine:
assignee: nobody → Vincent Ladeuil (vila)
importance: Undecided → Critical
milestone: none → phase-0
status: New → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.