Handle the skip return code in kernel_selftests on Bionic
| Affects | Status | Importance | Assigned to | Milestone | |
|---|---|---|---|---|---|
| ubuntu-kernel-tests |
Fix Released
|
Undecided
|
Po-Hsu Lin | ||
| linux (Ubuntu) |
Fix Released
|
Undecided
|
Unassigned | ||
| Trusty |
Invalid
|
Undecided
|
Unassigned | ||
| Xenial |
Invalid
|
Undecided
|
Unassigned | ||
| Bionic |
Fix Released
|
Undecided
|
Po-Hsu Lin | ||
Bug Description
== Justification ==
In the ubuntu_
(KSFT_SKIP).
However the code to handle this non-zero return code was not implemented
in the kselftest framework. And this will generate some false-positive
results as those skipped tests were treated as a failure.
For example the raw_skew test in timers on 4.15:
1..0 # Skipped: The clock was adjusted externally. Shutdown NTPd or other time sync daemons
not ok 1..7 selftests: raw_skew [FAIL]
And the test_user_copy.sh test in user (failure could be found in KVM kernels):
user: module test_user_copy is not found [SKIP]
not ok 1..1 selftests: test_user_copy.sh [FAIL]
Xenial is affected by this issue as well. Considering the fact that this issue in Bionic affects more tests / kernel variants, and it's way easier to fix this in Bionic, I decided to go for Bionic first.
== Fix ==
771cbc3b (selftests: lib.mk set KSFT_TAP_LEVEL to prevent nested TAP headers)
a3355440 (selftests: Fix lib.mk run_tests target shell script)
42b44c34 (selftests: lib.mk: cleanup RUN_TESTS define and make it readable)
3f4435b5 (selftests: lib.mk: add SKIP handling to RUN_TESTS define)
All these can be cherry-picked into Bionic.
The crucial patch is commit 3f4435b5, the other three patches are just
some minor fixes that pave the road for it to be cherry-picked.
== Test ==
Tested on a Bionic amd64 node, the result shows positive.
A test summary before the patch:
END GOOD ubuntu_
END GOOD ubuntu_
END GOOD ubuntu_
END GOOD ubuntu_
END GOOD ubuntu_
END GOOD ubuntu_
END GOOD ubuntu_
END GOOD ubuntu_
END GOOD ubuntu_
END GOOD ubuntu_
END ERROR ubuntu_
END GOOD ubuntu_
END GOOD ubuntu_
END GOOD ubuntu_
A test summary after the patch:
END GOOD ubuntu_
END GOOD ubuntu_
END GOOD ubuntu_
END GOOD ubuntu_
END GOOD ubuntu_
END GOOD ubuntu_
END GOOD ubuntu_
END GOOD ubuntu_
END GOOD ubuntu_
END GOOD ubuntu_
END GOOD ubuntu_
END GOOD ubuntu_
END GOOD ubuntu_
END GOOD ubuntu_
No tests were skipped unintentionally. And the raw_skew test in timers
is now marked as [SKIPPED] and no longer causing failure.
A complete test report could be found in the bug report attachment.
== Regression Potential ==
Low, this patch set just added support to handle the skip return code
and changes are limited to the kselftest framework itself, no actual
impact to normal users.
== Original bug report ==
In the selftests tool, we are using a KSFT_SKIP=4 return code for those skipped tests in the latest upstream kernel tree.
But the corresponding code to take care of this return value does not exist in Bionic and Xenial kernel, this will cause the test still being marked as failed:
selftests: test_user_copy.sh
=======
user: module test_user_copy is not found [SKIP]
not ok 1..1 selftests: test_user_copy.sh [FAIL]
| Changed in linux (Ubuntu): | |
| assignee: | nobody → Po-Hsu Lin (cypressyew) |
| Ubuntu Kernel Bot (ubuntu-kernel-bot) wrote Missing required logs. | #1 |
| Changed in linux (Ubuntu): | |
| status: | New → Incomplete |
| Changed in linux (Ubuntu): | |
| status: | Incomplete → In Progress |
| Po-Hsu Lin (cypressyew) wrote Re: Handle the skip return code in kernel_selftests | #2 |
| Changed in linux (Ubuntu): | |
| status: | In Progress → Confirmed |
| assignee: | Po-Hsu Lin (cypressyew) → nobody |
| tags: | added: ubuntu-kernel-selftests |
| tags: | added: bionic trusty |
| tags: | added: xenial |
| Po-Hsu Lin (cypressyew) wrote | #3 |
| Changed in linux (Ubuntu Trusty): | |
| status: | New → Invalid |
| tags: | added: sru-20190902 |
| tags: | added: linux-kvm |
| Changed in linux (Ubuntu Bionic): | |
| assignee: | nobody → Po-Hsu Lin (cypressyew) |
| status: | New → In Progress |
| Changed in ubuntu-kernel-tests: | |
| assignee: | nobody → Po-Hsu Lin (cypressyew) |
| status: | New → In Progress |
| Po-Hsu Lin (cypressyew) wrote | #6 |
| summary: |
- Handle the skip return code in kernel_selftests + Handle the skip return code in kernel_selftests on Bionic |
| Changed in linux (Ubuntu Xenial): | |
| status: | New → Invalid |
| Changed in linux (Ubuntu): | |
| status: | Confirmed → Fix Released |
| description: | updated |
| Changed in linux (Ubuntu Bionic): | |
| status: | In Progress → Fix Committed |
| Ubuntu Kernel Bot (ubuntu-kernel-bot) wrote | #8 |
| tags: | added: verification-needed-bionic |
| Po-Hsu Lin (cypressyew) wrote | #9 |
| Changed in ubuntu-kernel-tests: | |
| status: | In Progress → Fix Released |
| tags: |
added: verification-done-bionic removed: verification-needed-bionic |
| tags: | removed: xenial |
| Launchpad Janitor (janitor) wrote | #10 |
| Changed in linux (Ubuntu Bionic): | |
| status: | Fix Committed → Fix Released |
This bug is missing log files that will aid in diagnosing the problem. While running an Ubuntu kernel (not a mainline or third-party kernel) please enter the following command in a terminal window:
apport-collect 1812352
and then change the status of the bug to 'Confirmed'.
If, due to the nature of the issue you have encountered, you are unable to run this command, please add a comment stating that fact and change the bug status to 'Confirmed'.
This change has been made by an automated script, maintained by the Ubuntu Kernel Team.