Mutation Testing

Mutation testing is one of the least used software testing techniques. The basic idea behind mutation testing is quite simple. Mutation testing is a way to determine how effective a test suite (collection of individual test cases) is. You "mutate" the code under test, for example by changing a ">=" to a "<" and then run the so-called mutant against your test suite. If a new test case failure appears, then that is a good sign and means your test suite does in fact test the mutated code. But if no new test case failures appear, then that is a bad sign and probably means your test suite does not exercise the code path containing the mutated code, and so you will want to create new test cases. The term "mutation testing" is something of a misnomer; it might better be called "mutation analysis" because the result is a measure of test suite effectiveness rather than a pass/fail result.
Mutation testing has been made dramatically easier to perform in a .NET environment. This is due mostly to the fact that instead of parsing and recompiling souce code (which is very difficult), you can parse and assemble .NET Intermediate Language (which is significantly easier). Mutation testing is closely related to Code Coverage, a technique which measures which parts (either functions, blocks, or individual statements) of your code under test are touched by your test suite.
You can see a complete .NET mutation testing system written in C#, in the April 2006 issue of MSDN Magazine at .
This entry was posted in Software Test Automation. Bookmark the permalink.