Comparing Observed Values with Expected Values



My last two blog entries described two helper methods which programmatically compute Chi-square values and Normal values. I wrote them to create a C# method which compares a set of observed values with a set of expected values using the G test. The G test is very similar to the Chi-square test that you may have encountered in a Stats class in school. Here’s an example of how my G test code is called:

 

Console.WriteLine(“\nBegin G test demo”);

double[] observedValues = new double[] { 44, 56 };

double[] expectedValues = new double[] { 50, 50 };

Console.WriteLine(“observed = ” + observedValues[0] +

  ” ” + observedValues[1]);

Console.WriteLine(“expected = ” + expectedValues[0] +

  ” ” + expectedValues[1]);

 

string acceptOrReject = G_Test(observedValues, expectedValues, 0.05);

Console.WriteLine(“Result of G test = ” + acceptOrReject);

if (acceptOrReject == “accept”)

  Console.WriteLine(“Observed and expected match”);

else if (acceptOrReject == “reject”)

  Console.WriteLine(“Observed and expected are different\n”);

 

The output would be:

 

Begin G test demo

observed = 44 56

expected = 50 50

Result of G test = ‘accept’

Observed and expected match

 

The demo represents a hypothetical experiment where there are 100 trials of flipping a coin and so the expected values are 50 heads and 50 tails. A coin was actually flipped and produced 44 heads and 56 tails. The G test yielded an ‘accept’ of the null hypothesis at the 5% significance level so we conclude that there’s not enough evidence to say the coin data is biased so we loosely say observed and expected data match. If the result had been ‘reject’ we would have concluded that there was less than a 5% chance the results occurred by chance and so say the observed and expected data do not match. Here’s my G test method:

 

public static string G_Test(double[] observedValues,
  double[] expectedValues,
  double significanceLevel)
{
  // G = 2 * ∑ [O * ln(O/E)]
  // observed values are typically int counts (so input must be cast),
  // expected values are often non-ints because they are computed.
  // returns “accept” if null H is accepted: not enough evidence
  // to say observed != expected, or loosely, observed == expected
  // returns “reject” if null H rejected: loosely, observed != expected.
  // No Yates or Williams corrections. No check for low (<5) expected values.

  if (observedValues.Length < 2)
    throw new Exception(“there must be at least 2 observed values in G_Test()”);
  if (observedValues.Length != expectedValues.Length)
    throw new Exception(“obseved and expected must have same number of values”);

  double sumOfObervedValues = 0;
  double sumOfExpectedValues = 0;
  for (int i = 0; i < expectedValues.Length; ++i )
  {
    if (expectedValues[i] == 0.0)
      throw new Exception(“all expected values must be non-zero in G_Test()”);
      // consider an epsilon approach to avoid type double == comparison
    sumOfObervedValues += observedValues[i];
    sumOfExpectedValues += expectedValues[i];
  }
  if (sumOfObervedValues != sumOfExpectedValues)
    throw new Exception(“sum of observed value must equal sum of expected values);
 
  double sigma = 0.0;
  for (int i = 0; i < observedValues.Length; ++i)
  {
    double right = Math.Log(observedValues[i] / expectedValues[i]);
    double product = observedValues[i] * right;
    sigma += product;
  }

  double g = 2 * sigma;

  //Console.WriteLine(g);
  // consider refactoring this routine to return the p value
  // instead of “accept” or “reject”
  double prob = ChiSquare(g, observedValues.Length – 1);
  // length – 1 == df for ChhiSquare
  if (prob < significanceLevel)
    return “reject”;
    // reject null hypothesis => oberserved and expected are NOT equal
  else
    return “accept”;
   // accept null hypothesis => observed and expected are equal
   // (well, really not enough evidence to conclude they’re not equal)
}
 

 

About these ads
This entry was posted in Software Test Automation. Bookmark the permalink.