Pair-Wise Testing

Pair-wise testing is a simple and fundamental testing concept that has been around for a long tikme, and which is best explained by example. Suppose you are testing some application which has two drop-down controls and a radio-button list. The first drop-down control can have a value of either "a", or "b". The second-drop-down can have a value of "c", "d", "e", or "f". The radio-button can be either "g", "h", or "i". Exhaustively testing this situation is easily possible. The 2 * 4 * 3 = 24 test inputs would be:
a, c, g
a, c, g
a, c, i
a, d, g
. . .
b, f, i
But what if you had say, 8 controls, each of which could be one of 5 values. You’d have 5 * 5 * 5 * 5 * 5 * 5 * 5 * 5 = 390,625 combinations to test. The basic idea of pair-wise testing is to reduce the number of combinations in some way. This is accomplished a generating test sets which capture all possible pairs of inputs. In the example above, there are a total of 26 distinct pairs of inputs: {a,c} {a,d} (a,e} {a,f} {b,c} {b,d} (b,e} {b,f} {c,g} {c,h} {c,i} {d,g} {d,h} {d,i} {e,g} {e,h} {e,i} {f,g} {f,h} {f,i} {a,g} {a,h} {a,i}.
The theory is that these will generate good test case inputs. So consider the test input set {b, d, h}. This test set covers 3 of the 26 total possible pairs: {b,d} {b,h} {d,h}. Now suppose I add another test set of {b, c, h}. This second test set covers pairs {b,c} {b,h} {c,h}. Notice the {b,h} pair is covered by both test sets so at this point I have 5 of the 26 total pairs covered.
So, just how do you generate a list of test sets which capture all possible pairs? It turns out that this is a fascinating, NP-complete (translation, really, really hard) problem. The PICT (Pairwise Independent Combinatorial Testing) tool (see is a really nice command-line tool that can generate pair-wise combinations.
I wrote my own version of PICT for fun, using C#, and it worked out pretty well but generated about 15% more test sets than PICT. I’m experimenting with a genetic algorithm approach too.
This entry was posted in Software Test Automation. Bookmark the permalink.