Parallel tests add all tests to one partition if test durations are 0

Bug #914359 reported by Gary Poster
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Launchpad itself
Triaged
Low
Unassigned
Testrepository
Fix Released
Critical
Unassigned

Bug Description

This is really only particularly important if tests are miscalculated to take 0 seconds, which they have in the past.

In that case, all tests can be put in a single partition.

AFAIK, this is the sort of thing I think we want, modulo prejudices against lambda.

=== modified file 'testrepository/testcommand.py'
--- testrepository/testcommand.py 2011-11-01 15:24:36 +0000
+++ testrepository/testcommand.py 2012-01-10 16:34:01 +0000
@@ -208,10 +208,12 @@
         # Scheduling is NP complete in general, so we avoid aiming for
         # perfection. A quick approximation that is sufficient for our general
         # needs:
- # sort the tests by time
+ # sort the tests by time, and then by len of the partition's tests.
         # allocate to partitions by putting each test in to the partition with
         # the current lowest time.
- queue = sorted(timed.items(), key=operator.itemgetter(1), reverse=True)
+ queue = sorted(timed.items(),
+ key=lambda item: (item[1], -len(item[0])),
+ reverse=True)
         for test_id, duration in queue:
             timed_partitions[0][0] = timed_partitions[0][0] + duration
             timed_partitions[0][1].append(test_id)

Tags: paralleltest

Related branches

Gary Poster (gary)
Changed in launchpad:
status: New → Triaged
importance: Undecided → Low
tags: added: paralleltest
Changed in testrepository:
status: New → Triaged
importance: Undecided → Critical
summary: - Parallel tests when partitions have equal times can fail to divide tests
- equally
+ Parallel tests add all tests to one partition if test durations are 0
Revision history for this message
Robert Collins (lifeless) wrote :

For clarity, LP suffered from the 'tests get 0 duration' bug in an older testr; this led to a dual-core machine having a partition like:
[
  [0][testa, testb]
]
rather than
[
 [0][testa],
 [0][testb],
]
which would parallelise better in this corner case.

Revision history for this message
Robert Collins (lifeless) wrote :

Fixed in trunk. The sort that needed changing was the second one - the first sort sorts the tests to be in descending duration (so large chunks of time are allocated first), the second one in the inner loop selects the next partition to allocate to.

Changed in testrepository:
status: Triaged → Fix Committed
Changed in testrepository:
milestone: none → next
Changed in testrepository:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.