Classification and Prediction using Adaptive Boosting

I wrote an article titled “Classification and Prediction using Adaptive Boosting” that appears in the April 2013 issue of MSDN Magazine. See Adaptive boosting is a technique that combines a bunch of simple rules extracted from data to come up with a consensus prediction. Adaptive boosting is one form of what is usually called ensemble classification.

In the article I set up a demo where the goal is to predict whether a football team will win or lose an upcoming game, based on three simple rules culled from historical data. For example, suppose the goal is to predict if the team in question will win if their opponent is Detroit, they are playing at Home, and the Vegas point spread is Small. Suppose that historically, simple rules indicate that if the opponent is Detroit, the team will win. If the field is Home, the team will win. If the point spread is Small, the team will lose. Note that two of simple rules say the team will win, and one simple rule says the team will lose.

Adaptive boosting assigns a weight to each simple rule. In this case the three weights turn out to be 0.63, 3.15, and 4.49. Combining the weights yields a final prediction that the team will lose. Even though two of the three rules predict a win, the higher weight of the third rule overcomes the first two rules and gives an overall prediction of lose. Determining the weights of the simple rules is the heart of the adaptive boosting algorithm.

To be honest, I’m not entirely convinced about how effective adaptive boosting is. There are other ensemble machine learning algorithms, such as random forests, that I suspect may be superior to adaptive boosting. Regardless, adaptive boosting is a more or less standard machine learning technique.


This entry was posted in Machine Learning. Bookmark the permalink.