dashboard pass-rates can be misleading
| Affects | Status | Importance | Assigned to | Milestone | |
|---|---|---|---|---|---|
| Ubuntu CI Services |
New
|
Medium
|
Unassigned | ||
Bug Description
Taking this as an example:
http://
we have an 80% "passrate". However, the one failure is super critical - we phablet-test-run didn't actually run the tests.
I'm not sure where we should fix this issue. The code running the tests won't know how many tests should have been run, so it can't insert a bunch of failures by hand. I think we might have to somehow do something special in the dashboard code flag SmokeResults with a failed phablet-test-run as 0%. That might be tricky as we take some fairly optimized SQL approaches for determine pass-rates efficiently. Something like this could cause sql queries inside inner-loops of large data-sets.
Maybe I'm looking at this in the wrong way and there's an simpler approach for handling this?
| Changed in ubuntu-ci-services-itself: | |
| importance: | Undecided → Medium |
| Robert Bruce Park (robru) wrote | #1 |
Maybe the dashboard should just be adjusted so that 0%-85% shows up in red, 85%-99% shows up in yellow, and 100% is green?