Checkbox only reports the first resource found for a job requirement

Bug #560948 reported by Marc Tardif
12
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Checkbox
Fix Released
Undecided
Marc Tardif
checkbox (Ubuntu)
Fix Released
Undecided
Marc Tardif
Lucid
Fix Released
Undecided
Unassigned

Bug Description

The impact of this bug is that when test results are reported, partial device information might be linked to the results. This fix makes sure that all related devices are properly linked to the test results.

Checkbox uses job requirements in order to link resources to results. For example, the following requirement should like a result to all the network devices found on the system:

  device.category == 'NETWORK'

So, if there are two network devices, such as an ethernet and a wireless controller, then these should be linked to the result. However, the checkbox.resource module seems to bail as soon as the first resource is found.

Note that the requirements assessment is still correct, whether to run a job or not. However, the details of the requirements are simply missing information.

Related branches

Marc Tardif (cr3)
Changed in checkbox:
assignee: nobody → Marc Tardif (cr3)
status: New → In Progress
status: In Progress → Fix Committed
Marc Tardif (cr3)
Changed in checkbox:
status: Fix Committed → Fix Released
Marc Tardif (cr3)
Changed in checkbox (Ubuntu Lucid):
status: New → In Progress
milestone: none → lucid-updates
Changed in checkbox (Ubuntu):
status: New → Fix Released
assignee: nobody → Marc Tardif (cr3)
Marc Tardif (cr3)
description: updated
Revision history for this message
Daniel Manrique (roadmr) wrote :
Download full text (3.3 KiB)

Hello, the following information should complete this bug as a valid SRU report, to hopefully upload this fix to Lucid.

Solution:

Checbox revision 786 adds code to ensure that all related devices are properly linked to the test results.

TEST CASE:

We need to be tricky when running this test case, as checkbox 0.9.1 doesn't log detailed information about each test that we can access after a test run.

1- Launch System Testing on a system with Ubuntu 10.04 and two network controllers (eth0 and eth1 for instance). A virtual machine with two interfaces works fine for this test.
2- On the test selection screen, press "Deselect All", then expand "Network tests" and select only "network_test". Press Next.
3- When the test runs, confirm that it detected both network controllers. DO NOT PRESS NEXT.
4- Here's the tricky part: close checkbox by pressing the X on the top left corner. This forces it to log an exception for the execution interruption, and in this case it *does* log the information we need.

Now look at the log file (~/.cache/checkbox/checkbox.log), near the end there's a log line starting with ERROR.

Without the fix:

Keyboard interrupt while running event handler None.ManualTest.prompt_manual() for event type 'prompt-manua
l' with args (<checkbox_gtk.gtk_interface.GTKInterface object at 0xb7701eec>, {'status': 'uninitiated', 'description': u'Detecting your netw
ork controller(s):\n\n$output\n\nIs this correct?', 'plugin': 'manual', 'requires': ["device.category == 'NETWORK'"], 'command': 'network_te
st', 'suite': 'network', 'type': 'test', 'resources': [{'category': 'NETWORK', 'subproduct_id': '30', 'product_id': '4110', 'bus': 'pci', 'v
endor_id': '32902', 'driver': 'e1000', 'path': '/devices/pci0000:00/0000:00:03.0', 'subvendor_id': '32902'}], 'name': 'network_test'}) {}.

With the fix:

Keyboard interrupt while running event handler None.ManualTest.prompt_manual() for event type 'prompt-manual' with args (<checkbox_gtk.gtk_interface.GTKInterface object at 0xb76b9eec>, {'status': 'uninitiated', 'description': u'Detecting your network controller(s):\n\n$output\n\nIs this correct?', 'plugin': 'manual', 'requires': ["device.category == 'NETWORK'"], 'command': 'network_test', 'suite': 'network', 'type': 'test', 'resources': [{'category': 'NETWORK', 'subproduct_id': '30', 'product_id': '4110', 'bus': 'pci', 'vendor_id': '32902', 'driver': 'e1000', 'path': '/devices/pci0000:00/0000:00:03.0', 'subvendor_id': '32902'}, {'category': 'NETWORK', 'subproduct_id': '30', 'product_id': '4110', 'bus': 'pci', 'vendor_id': '32902', 'driver': 'e1000', 'path': '/devices/pci0000:00/0000:00:08.0', 'subvendor_id': '32902'}], 'name': 'network_test'}) {}.

Notice how the resources array contains data for the two network cards with the fixed code, whereas the un-fixed version only shows the first card.

Regression potential:

The logic changes in themselves are quite harmless. They will potentially return larger sets of data, so if other parts of checkbox aren't ready to deal with this data, that might entail failures where there were previously none. However this hasn't happened in real-world usage of this code, and no subsequent bug fixes are related to r...

Read more...

Revision history for this message
Ara Pulido (ara) wrote :

This bug is awaiting verification that the checkbox version in lucid-proposed solves the problem. Please test checkbox and update this bug with the results. If the problem is solved, change the tag 'verification-needed' to 'verification-done'.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

tags: added: verification-needed
Revision history for this message
Martin Pitt (pitti) wrote : Please test proposed package

Hello Marc, or anyone else affected,

Accepted checkbox into lucid-proposed, the package will build now and be available in a few hours. Please test and give feedback here. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you in advance!

Changed in checkbox (Ubuntu Lucid):
status: In Progress → Fix Committed
Revision history for this message
Chad A. Davis (chadadavis) wrote :

Verified fix from checkbox 0.9.2 on Lucid

tags: added: verification-done
removed: verification-needed
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package checkbox - 0.9.2

---------------
checkbox (0.9.2) lucid-proposed; urgency=low

  New upstream release (LP: #567568):
  * Added referer when sending submissions to Launchpad (LP: #550973)
  * Added suggests to checkbox package in debian/control file (LP: #352740)
  * Fixed udev_resource script to be more resilient (LP: #556824)
  * Fixed cdimage_resource script to read casper.log (LP: #558728)
  * Fixed reporting all resources found for a job (LP: #560948)
  * Fixed stalling when using kdesudo to start backend (LP: #557443)
  * Fixed starting the appropriate default browser on UNR (LP: #563050)
  * Fixed opening the report with the gconf preferred browser (LP: #562580)
  * Fixed suspend_test to use relative time for wakealarm (LP: #349768)
  * Fixed backend not getting terminated upon closing (LP: #553328)
 -- Daniel Manrique <email address hidden> Wed, 22 Jun 2011 14:18:08 -0400

Changed in checkbox (Ubuntu Lucid):
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.