Test Case Input Symmetry

I ran across an interesting effect recently. The effect is related to symmetry in test case input. The bottom line is that when you are testing you cannot assume the sending inputs (a,b) to some system under test is the same as sending inputs (b,a) to the system. This is true even when the system under test should treat inputs the same, for example, a sum(x,y) function. I was looking at a Poker Library. One of the methods in the library is a Hand.Compare(h1,h2) which compares poker hands h1 and h2, returning -1 if hand h1 is worse than hand h2, returning 0 if the hands tie, and returning +1 if hand h1 is better than hand h2. For example Hand.Compare("Ac Ad Ah 9c 9d", "Kc Kd Qh Qs 2c") returns +1 because a full house aces over nines is better than two pair kings over queens. When testing such a method you cannot assume that if this test case passed, that Hand.Compare("Kc Kd Qh Qs 2c", "Ac Ad Ah 9c 9d"), that is the same inputs but with order reversed, with an expected value of -1 would also pass. This principle — that you must test all permutations of input arguments — is pretty well known, but I see even experienced testers neglect it every now and then.
This entry was posted in Software Test Automation. Bookmark the permalink.