Commercial Test Automation Frameworks

There are four main approaches to software test automation: commercial test frameworks, Open Source test frameworks, lightweight custom test harnesses, and heavyweight custom test harnesses. There are a huge number of commercial test automation frameworks available. Some of the larger and better known companies which sell these often very expensive programs include Mercury, Segue, Compuware, and Rational. Each of these companies sells many different tools. For example, Mercury sells QuickTest for basic unit testing, and WinRunner for Web application testing. Segue sells SilkTest for basic unit testing and SilkPerformer for load testing. Compuware sells QACenter Enterprise Edition for functional testing, and QACenter Performance Edition for performance testing. Rational sells Rational Robot for general purpose client-server testing, and Rational Functional Tester for windows-based application testing. A once very popular tool which is no longer being sold was Visual Test. All of these products are fairly similar. Click on the image at the bottom of this blog entry to see a screenshot of the old Visual Test.

I have used products from all these companies. In general, the main advantage of using commercial test automation tools is that they provide you with a coherent management structure (although you often have to buy a separate product). An alternative to using commercial test automation frameworks is simply to write your own custom test harnesses. Compared to custom test harnesses, commercial test frameworks are extremely expensive. Commercial automation frameworks have a very steep learning curve (some even have a proprietary scripting language you must learn, and/or a complex object to learn on top of a not-so-obvious user interface). Commercial tools are constantly being revised and repackaged, and even the companies themselves are often acquired by other companies. For example, Visual Test was originally developed as an internal Microsoft tool, then marketed by Microsoft as a commercial product.  Visual Test was then acquired by the Rational company, and then Rational was acquired by IBM. But by far the biggest disadvantage of commercial automation frameworks is a subjective issue.  If you use a commercial tool, you tend to exercise your system under test based on what the test tool can do most easily, rather than testing the SUT based on what needs to be tested.

Based on my experience, the most successful testing approach is to use both commercial test automation tools (if you can afford them) and lightweight custom test automation. Or put another way, there is no single best approach to software test automation.

This entry was posted in Software Test Automation. Bookmark the permalink.

One Response to Commercial Test Automation Frameworks

  1. Rafa says:

    Hello James, I have to work with QACenter, but I can´t find any manual about this tool in spanish(i´m spanish and my englihs is very bad)
    ¿do you know anythig?

Comments are closed.