Rule Sets and Automatic Software Test Case Generation

The other day I had a strange idea about a possible way to automatically generate test case data. Automated test case generation is one of the Holy Grails of testing. It is relatively easy to generate test case input but very  difficult to generate expected results. One approach is to feed random input to a system under test to try and produce a fatal state; this technique doesn’t need an expected result (the implied result is that he SUT doesn’t crash). Another approach to automated software test case generation is to decorate the system under test with all kinds of pre-conditions and post-conditions, and then have some relatively sophisticated tool generate both test case inputs and expected values from the pre- and post- condition information. See for example Pex at Another approach is various forms of model based development. For example, you create some model using UML (which contains pre-conditions and post-conditions) or similar meta-language which in turn can generate source code and test cases. Anyway, my idea is to use an evolutionary algorithm approach for automated test case generation for certain types of systems. Imagine a SUT with four input parameters, A, B, C, D. Input parameter A is discrete and can take on one of three values: a1, a2, a3. Similarly, B can be b1, b2, b3. C can be c1, c2. D can be d1, d2. Suppose the SUT can transition to one of three states, s1, s2, s3. There are a total of 3 * 3 * 2 * 2 = 36 inputs which could easily be handled but I’ll illustrating with a small example. Now consider a rule based system. One rule might be "if a1 and c2 then s3". There are a total of 131 possible rules for this example. My idea is to manually list a relatively small set of training rules, and then use an evolutionary algorithm such as a Genetic Algorithm or Simulated Annealing to produce a set of rules from all possible rules. These rules could then be used to generate test case expected results. The results wouldn’t be perfect; more along the line of input = {a1, b2, c1, d2}, output = {s2} => possible test case failure according to generated rule set. I have no idea if anyone has investigated such an idea before.
This entry was posted in Software Test Automation. Bookmark the permalink.