testr run doesn't complain when filtering results in no selected tests

Bug #1264274 reported by Thomas Herve
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Testrepository
Triaged
Wishlist
Unassigned

Bug Description

If I do:

$ testr run mypackage.mydule_typo

I don't get any error from testr run, and also a success error code. It seems to somewhat be by design, because testr lists all the tests available, and then filter with what I pass on the command line. In this case though I pass directly a module name, so it'd be nice if 1) It just tried to import it and 2) it told me that it failed to do so.

Revision history for this message
Jaspreet (walia-jaspreet) wrote :

I agree with Thomas Herve suggestion. I ran the below non existing test case and got the success message, however I didn't get test count(since there was no test run) .

It would be nice if it could inform user about non-existent tests.

I ran two set of tests. With non existing tests I got following message
PASSED (id=5)

With existing tests I got
Ran 8 tests in 7.346s (+2.018s)
PASSED (id=4)

Please find below tests I ran
-----------------------------------------------------------------
# 1 . testr run tempest.api.compute.flavors.test_flavors_negative1
running=OS_STDOUT_CAPTURE=${OS_STDOUT_CAPTURE:-1} \
OS_STDERR_CAPTURE=${OS_STDERR_CAPTURE:-1} \
OS_TEST_TIMEOUT=${OS_TEST_TIMEOUT:-500} \
OS_TEST_LOCK_PATH=${OS_TEST_LOCK_PATH:-${TMPDIR:-'/tmp'}} \
${PYTHON:-python} -m subunit.run discover -t ./ ${OS_TEST_PATH:-./tempest/test_discover} --list
-------------
PASSED (id=5)
-----------

Ran the following test , run ran successfully along with number of tests it ran .
# testr run tempest.api.compute.flavors.test_flavors_negative

running=OS_STDOUT_CAPTURE=${OS_STDOUT_CAPTURE:-1} \
OS_STDERR_CAPTURE=${OS_STDERR_CAPTURE:-1} \
OS_TEST_TIMEOUT=${OS_TEST_TIMEOUT:-500} \
OS_TEST_LOCK_PATH=${OS_TEST_LOCK_PATH:-${TMPDIR:-'/tmp'}} \
${PYTHON:-python} -m subunit.run discover -t ./ ${OS_TEST_PATH:-./tempest/test_discover} --list
running=OS_STDOUT_CAPTURE=${OS_STDOUT_CAPTURE:-1} \
OS_STDERR_CAPTURE=${OS_STDERR_CAPTURE:-1} \
OS_TEST_TIMEOUT=${OS_TEST_TIMEOUT:-500} \
OS_TEST_LOCK_PATH=${OS_TEST_LOCK_PATH:-${TMPDIR:-'/tmp'}} \
${PYTHON:-python} -m subunit.run discover -t ./ ${OS_TEST_PATH:-./tempest/test_discover} --load-list /tmp/tmpKZMyg_
Ran 8 tests in 7.346s (+2.018s)
PASSED (id=4)

Revision history for this message
Robert Collins (lifeless) wrote :

I'm not sure what to do about this - its certainly a valid complaint, but I worry about the difference between None and 0 here - basically filtering down to nothing to do is a legitimate thing. Note that the arguments are regexes, so a) you can be a lot more pithy with your selectors and b) we don't know (and currently can't) whether something is a module to import or whatnot - testr is multi-language.

Changed in testrepository:
status: New → Triaged
summary: - testr run doesn't complain when it doesn't run tests
+ testr run doesn't complain when filtering results in no selected tests
Changed in testrepository:
importance: Undecided → Wishlist
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.