I wrote an article titled “Regression Using PyTorch New Best Practices, Part 2: Training, Accuracy, Predictions” in the November 2022 edition of Microsoft Visual Studio Magazine. See https://visualstudiomagazine.com/articles/2022/11/14/pytorch-regression-2.aspx.

The article is the second of two articles that explain how to create and use PyTorch regression model. A regression problem is one where the goal is to predict a single numeric value. For example, you might want to predict the annual income of a person based on their sex, age, state where they live and political leaning (conservative, moderate, liberal).

The demo data looks like:

1 0.24 1 0 0 0.2950 0 0 1 -1 0.39 0 0 1 0.5120 0 1 0 1 0.63 0 1 0 0.7580 1 0 0 -1 0.36 1 0 0 0.4450 0 1 0 . . .

The fields are sex (male = -1), age (divided by 100), State (Michigan = 100, Nebraska = 010, Oklahoma = 001), income (divided by 100,000) and political leaning (conservative = 100, moderate = 010, liberal = 001). The goal is to predict income.

The design of the demo network is:

import torch as T class Net(T.nn.Module): def __init__(self): super(Net, self).__init__() self.hid1 = T.nn.Linear(8, 10) # 8-(10-10)-1 self.hid2 = T.nn.Linear(10, 10) self.oupt = T.nn.Linear(10, 1) T.nn.init.xavier_uniform_(self.hid1.weight) T.nn.init.zeros_(self.hid1.bias) T.nn.init.xavier_uniform_(self.hid2.weight) T.nn.init.zeros_(self.hid2.bias) T.nn.init.xavier_uniform_(self.oupt.weight) T.nn.init.zeros_(self.oupt.bias) def forward(self, x): z = T.tanh(self.hid1(x)) z = T.tanh(self.hid2(z)) z = self.oupt(z) # regression: no activation return z

The code to train the network is:

def train(model, ds, bs, lr, me, le): # dataset, bat_size, lrn_rate, max_epochs, log interval train_ldr = T.utils.data.DataLoader(ds, batch_size=bs, shuffle=True) loss_func = T.nn.MSELoss() optimizer = T.optim.Adam(model.parameters(), lr=lr) for epoch in range(0, me): epoch_loss = 0.0 # for one full epoch for (b_idx, batch) in enumerate(train_ldr): X = batch[0] # predictors y = batch[1] # target income optimizer.zero_grad() oupt = model(X) loss_val = loss_func(oupt, y) # a tensor epoch_loss += loss_val.item() # accumulate loss_val.backward() # compute gradients optimizer.step() # update weights if epoch % le == 0: print("epoch = %4d | loss = %0.4f" % \ (epoch, epoch_loss))

The demo presents two different functions to compute model accuracy. One accuracy function iterates line by line which is slow but useful for analysis. The second accuracy function operates on an entire dataset in one pass which is fast but doesn’t lend itelf to debugging.

*A typical annual income in the U.S. in 1870 was about $700. In 2020 the typical income is about $70,000.*

*I collected coins when I was a young man. I was especially interested in two and three cent coins. Left: The two-cent coin was produced for circulation for only eight years, from 1864 to 1872. It was quite large at 23 mm — slightly larger than a modern nickel. Right: The three-cent coin was produced for 38 years, from 1851 to 1889. It was small at 14 mm — slightly smaller than a modern dime.*

You must be logged in to post a comment.