EqualConstraint provides inadequate failure information for IEnumerables

Bug #787106 reported by Charlie Poole
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
NUnit Framework
Fix Released
High
Charlie Poole
NUnit V2
Fix Released
High
Charlie Poole

Bug Description

When an EqualConstraint compares two IEnumerables and one of them is _not_ a collection, the failure reported inadequately. This was called to our attention by the following post on nunit-discuss by Gishu Pillai but applies more generally than the post indicates.

Given any CustomType containing a Data (int) property and Name
(string) property. Equals and ToString overridden

           var someArray = new[] { "A001", "B002", "C003" };
           CollectionAssert.AreEqual( new[] { "x", "Y", "Z" },
someArray);

 Expected and actual are both <System.String[3]>
 Values differ at index [0]
 Expected string length 1 but was 4. Strings differ at index 0.
 Expected: "x"
 But was: "A001"

-------------------------------------------------------------------------------------
Which is perfect. But now if any of the arguments to AreEqual are a
result of a Linq query, the failure message drops in readability
           var someArray = new[] { "A001", "B002", "C003" };
           CollectionAssert.AreEqual( new[] { "x", "Y", "Z" },

someArray.Select(item => item));

 Expected: < "x", "Y", "Z" >
 But was: <System.Linq.Enumerable
+WhereSelectArrayIterator`2[System.String,System.String]>

----------------------------------------------------------------------------------------
Even worse
              CollectionAssert.AreEqual( new []{'A', 'B',
'C'}.Select(item=>item.ToString()),
                                       new [] {"Apple", "Ball",
"Dog"}.Select(item=> item.Substring(0,1)));

 Expected: <System.Linq.Enumerable
+WhereSelectArrayIterator`2[System.Char,System.String]>
 But was: <System.Linq.Enumerable
+WhereSelectArrayIterator`2[System.String,System.String]>

if you change the 'C' to 'D' - the test passes as expected.

I've been trying to write something similar for my custom-adaptation
of Hamcrest and I think I see what is happening (an are-types-
compatible check failing for IEnumerable<T> and IEnumerable<K>) .
However if CollectionAssert.AreEqual internally uses a ToList() or
ToArray() before processing its arguments, I think the error messages
would be much more helpful.

Changed in nunitv2:
status: New → Triaged
importance: Undecided → High
assignee: nobody → Charlie Poole (charlie.poole)
milestone: none → 2.6.0b1
status: Triaged → Fix Committed
Revision history for this message
Charlie Poole (charlie.poole) wrote :

The fix in NUnit 2.6 is somewhat adhoc. NUnit 3.0 needs a better fix after the Constraint class refactoring is done.

Note that an IEnumerable may not be repeatable, so it's important to save the failing value rather than re-fetching it as we do in the case of a collection.

Changed in nunit-3.0:
status: New → Triaged
importance: Undecided → High
Changed in nunitv2:
status: Fix Committed → Fix Released
Changed in nunit-3.0:
milestone: none → 2.9.6
Changed in nunit-3.0:
assignee: nobody → Charlie Poole (charlie.poole)
Changed in nunit-3.0:
status: Triaged → Fix Committed
Changed in nunit-3.0:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.