One of my colleagues (Kirk O.) recently posed this challenge to me: create a neural network that can calculate the area of a triangle. I mentally scoffed. How hard could it be?
Well, several hours later, I had a working NN system that can calculate the area of a triangle but it was more difficult than I expected. The area of a triangle is one-half the base times the height. The “one-half” is just a constant so in reality the challenge is to compute f(x,y) = x * y or more generally, compute a polynomial.
I dove in a wrote some code but I really had to experiment with the hyperparameters, especially the number of hidden nodes to use, the learning rate, and the momentum factor. In the end, I used a 2-5-1 network — two input nodes for the base and the height, five hidden processing nodes, and a single output node for the area.
I used tanh for the hidden layer activation function. I didn’t need an output layer activation function because this is a regression problem. I had to use a very small learning rate (0.0001) and a momentum factor = 0.0 to get good results.
There’s always a problem trying to determine model accuracy when dealing with regression. Here I cheated a bit by counting an area that’s within 0.5 of the correct result as a correct prediction. I also fudged by limiting the base and height to be less than 10.
The demo NN got 91.70% accuracy on its 1000-item training data set, and then 97.00% accuracy on a 100-item test set. It’s unusual to have better accuracy on the test data than on the training data but with NNs weird things happen.
So, thanks Kirk, for making me toss a couple hours of my life away. No, actually, it was really a fun little problem and I learned some valuable tricks.