I wrote an article titled “Regression Using PyTorch, Part 1: New Best Practices” in the November 2022 edition of Microsoft Visual Studio Magazine. See https://visualstudiomagazine.com/articles/2022/11/01/pytorch-regression.aspx.
A regression problem is one where the goal is to predict a single numeric value. For example, you might want to predict the annual income of a person based on their sex, age, state where they live and political leaning (conservative, moderate, liberal). In my article I explain how to prepare training data and how to design a PyTorch neural network for regression.
My demo begins by loading a 200-item file of training data and a 40-item set of test data. Each tab-delimited line represents a person. The fields are sex (male = -1, female = +1), age, state of residence, annual income and politics type. The goal is to predict income from sex, age, state, and political leaning.
After the training data is loaded into memory, the demo creates an 8-(10-10)-1 neural network. This means there are eight input nodes, two hidden neural layers with 10 nodes each, and one output node.
The demo uses a batch size of 10, Adam (adaptive momentum) optimization with a learning rate of 0.01, and maximum training epochs of 1,000 passes through the training data.
After 1,000 training epochs, the demo program computes the accuracy of the trained model on the training data as 91 percent (182 out of 200 correct). The model accuracy on the test data is 85 percent (34 out of 40 correct). For regression problem accuracy, you must specify how close a prediction must be to the true value in order to be counted as a correct prediction. For the demo, a predicted income that’s within 10 percent of the true value is counted as a correct prediction.
After evaluating the trained network, the demo predicts the income for a person who is male, 34 years old, from Oklahoma, who is a political moderate. The prediction is $45,392.60.
The neural network definition is:
class Net(T.nn.Module): def __init__(self): super(Net, self).__init__() self.hid1 = T.nn.Linear(8, 10) # 8-(10-10)-1 self.hid2 = T.nn.Linear(10, 10) self.oupt = T.nn.Linear(10, 1) T.nn.init.xavier_uniform_(self.hid1.weight) T.nn.init.zeros_(self.hid1.bias) T.nn.init.xavier_uniform_(self.hid2.weight) T.nn.init.zeros_(self.hid2.bias) T.nn.init.xavier_uniform_(self.oupt.weight) T.nn.init.zeros_(self.oupt.bias) def forward(self, x): z = T.tanh(self.hid1(x)) z = T.tanh(self.hid2(z)) z = self.oupt(z) # regression: no activation return z
Even though the code isn’t very long, there are a ton of details to understand.
In my mind, there’s a connection between producing something of value and income generation. Therefore, I’m always a bit uneasy when it comes to gambling games. But I enjoy the intellectual challenge of probability and prediction.