Software Risk Analysis

If you are a software tester, sooner or later you’ll have to deal with software risk analysis and management. Risk is loosely defined as the possibility that an unwanted event can happen. Risk analysis is the process of identifying and estimating the likelihood of possible risks. Here’s an example. First you list possible risk events. Risk analysis can happen at different levels. For example, at a management level, risk events are things like key personnel quitting. At an implementation level, risk events are things like choosing a multi-threaded architecture for improved performance vs. choosing a single-threaded architecture for simplicity. And as a tester, risk events are often not discovering a particular type of bug. Let’s suppose you are worried about 3 arbitrary risk events, A, B, C. Now you must estimate the probability of each of these events. Suppose you guess that the probability of risk events A, B, and C are 0.3, 0.4, and 0.1 respectively. Notice that because the risk events are not necessarily related, their probabilities need not sum to 1. Now you must estimate the risk impact of each risk event. Let’s imagine that you assign the costs of events A, B, and C as \$1000, \$3000, and \$6000 respectively. You can now compute the so-called risk exposure of each event, which is just the probability of the event times the impact:

A: (0.3)(\$1000) = \$300
B: (0.4)(\$3000) = \$1200
C: (0.1)(\$6000) = \$600

You can use this information to help prioritize where you spend your resources. Next you can compute your overall risk exposure, which is just the sum of the individual events’ exposures: \$300 + \$1200 + \$600 = \$2100. You can use this value to compare against previous risk analysis data from other projects, and you can use the value to estimate your likely cost overruns from your base budget.
I am not a fan of the risk analysis procedure described above. All the numbers give a very false sense of the accuracy and precision of the process. Here’s an alternative approach I prefer. Instead of assigning numerical probabilities to each risk event, you assign a likelihood attribute of L, M, or H (for low, medium, and high). Then instead of assigning a numeric (money) cost for the impact, you assign one of the same L, M, H attributes. So, the example above might translate to the following where the first attribute is the likelihood, and the second attribute is the impact:

A: M x L -> ML
B: H x M -> HM
C: L x H -> LH

Each risk event will be described by one of 9 attribute pairs: LL, LM, LH, ML, MM, MH, HL, HM, HH. You should pay most attention to the HH and HM risk events. You can often ignore LL and LM risk events. How you treat the other five risk types will depend on your particular situation.