I wrote an article titled “Neural Regression using CNTK” in the June issue of Microsoft MSDN Magazine. See https://msdn.microsoft.com/en-us/magazine/mt846729.
The goal of a regression problem is to predict a single numeric value (such as the age of a person) — as opposed to a classification problem where the goal is to predict a discrete value (such as the sex of a person, male or female).
There are many different techniques that can be used to tackle a regression problem. You’re likely familiar with linear regression if you took a statistics class in college. In my article, I show how to do regression using a deep neural network with the CNTK library.
I always believe in presenting concrete examples, so in my article I demonstrated neural regression on the Yacht Hull Dataset. This dataset has six numeric predictor values that describe a boat hull (length-beam ratio, center of buoyancy, etc.) and the goal is to predict a measure of resistance of the hull shape.
For my demo, I created a 6-(5-5)-1 neural network, that is, one with six input nodes, two hidden layers each with five nodes, and one output node. I trained the model using the Adam optimizer, which is an advanced variation of simple stochastic gradient descent.
One difficulty with regression problems is that there’s no inherent definition of accuracy. For example, if a predicted hull resistance is 0.6000 and the actual hull resistance is 0.6543 is the prediction correct? For regression problems you must define a custom accuracy metric. For example, you might define an accurate prediction as one which is plus or minus 10 percent of the true value.