Code Coverage

Code coverage is a software testing activity which is intended to measure how effective your test effort is. In other words, code coverage does not directly test the program/system you are testing. Code coverage analysis is the process of monitoring which parts of your system under test (SUT) are exercised by your test suite. (A test suite is a collection of individual test cases). For example, suppose you have a test suite which consists of 10,000 test cases. And suppose your SUT has 250,000 statements. In a code coverage situation you would run your test suite. Now suppose that the tests touch 200,000 statements, meaning that 200,000 statements are executed during the test suite run. In this situation your code coverage is 200,000 / 250,000 = 80%. You now know that certain parts of your SUT aren’t being tested at all! So, you can create new test cases. It is not uncommon for a test team to be fairly confident that they have good test suites, but after a code coverage analysis they discover they only have around 60-70% coverage. This has happened to me before.

It should be clear that code coverage is an absolutely essential part of any significant test effort.

Code coverage can operate at different levels of granularity. For instance, you can perform code coverage at the statement level, checking what percent of SUT statements are exercised by your test suite. Or you can perform code coverage at the block level, checking what percent of the blocks of SUT code (typically code between { and } in languages such as C/C++, C#, and Java). Or you can perform code coverage at the function/method level, just checking to see what percentage of functions/methods in the SUT are entered.


Code coverage is simple in principle, but difficult in practice. There are several approaches. One approach is to instrument the source code of the SUT by inserting special code which logs entry and exit events as the code executes. This idea sounds good but doing this manually is very time-consuming, and instrumenting source code programmatically is extremely difficult — you’re essentially writing a complete language parser. Another approach is to create a service which monitors the run time activities of the machine/CPU running the tests. This is really, really difficult. In a .NET and Java environment you can attempt  to instrument the Intermediate Language (.NET) or Byte Code (Java) which is a tiny bit easier, but still really difficult. There are other approaches too but they’re all hard. I’m a big believer in lightweight, custom test automation, but code coverage is one area where using a commercial tool often makes sense.

You can read about code coverage in the April 2004 issue of MSDN Magazine at

If you enjoyed this blog entry, you might enjoy working at Microsoft. My parent company, Volt Information Sciences, is always looking for software engineers.  Check the job listings at

This entry was posted in Software Test Automation. Bookmark the permalink.