“Regression Using PyTorch New Best Practices, Part 2: Training, Accuracy, Predictions” in Visual Studio Magazine

I wrote an article titled “Regression Using PyTorch New Best Practices, Part 2: Training, Accuracy, Predictions” in the November 2022 edition of Microsoft Visual Studio Magazine. See https://visualstudiomagazine.com/articles/2022/11/14/pytorch-regression-2.aspx.

The article is the second of two articles that explain how to create and use PyTorch regression model. A regression problem is one where the goal is to predict a single numeric value. For example, you might want to predict the annual income of a person based on their sex, age, state where they live and political leaning (conservative, moderate, liberal).

The demo data looks like:

 1   0.24   1  0  0   0.2950   0  0  1
-1   0.39   0  0  1   0.5120   0  1  0
 1   0.63   0  1  0   0.7580   1  0  0
-1   0.36   1  0  0   0.4450   0  1  0
. . .

The fields are sex (male = -1), age (divided by 100), State (Michigan = 100, Nebraska = 010, Oklahoma = 001), income (divided by 100,000) and political leaning (conservative = 100, moderate = 010, liberal = 001). The goal is to predict income.

The design of the demo network is:

import torch as T

class Net(T.nn.Module):
  def __init__(self):
    super(Net, self).__init__()
    self.hid1 = T.nn.Linear(8, 10)  # 8-(10-10)-1
    self.hid2 = T.nn.Linear(10, 10)
    self.oupt = T.nn.Linear(10, 1)

    T.nn.init.xavier_uniform_(self.hid1.weight)
    T.nn.init.zeros_(self.hid1.bias)
    T.nn.init.xavier_uniform_(self.hid2.weight)
    T.nn.init.zeros_(self.hid2.bias)
    T.nn.init.xavier_uniform_(self.oupt.weight)
    T.nn.init.zeros_(self.oupt.bias)

  def forward(self, x):
    z = T.tanh(self.hid1(x))
    z = T.tanh(self.hid2(z))
    z = self.oupt(z)  # regression: no activation
    return z

The code to train the network is:

def train(model, ds, bs, lr, me, le):
  # dataset, bat_size, lrn_rate, max_epochs, log interval
  train_ldr = T.utils.data.DataLoader(ds, batch_size=bs,
    shuffle=True)
  loss_func = T.nn.MSELoss()
  optimizer = T.optim.Adam(model.parameters(), lr=lr)

  for epoch in range(0, me):
    epoch_loss = 0.0  # for one full epoch
    for (b_idx, batch) in enumerate(train_ldr):
      X = batch[0]  # predictors
      y = batch[1]  # target income
      optimizer.zero_grad()
      oupt = model(X)
      loss_val = loss_func(oupt, y)  # a tensor
      epoch_loss += loss_val.item()  # accumulate
      loss_val.backward()  # compute gradients
      optimizer.step()     # update weights

    if epoch % le == 0:
      print("epoch = %4d  |  loss = %0.4f" % \
        (epoch, epoch_loss))

The demo presents two different functions to compute model accuracy. One accuracy function iterates line by line which is slow but useful for analysis. The second accuracy function operates on an entire dataset in one pass which is fast but doesn’t lend itelf to debugging.



A typical annual income in the U.S. in 1870 was about $700. In 2020 the typical income is about $70,000.

I collected coins when I was a young man. I was especially interested in two and three cent coins. Left: The two-cent coin was produced for circulation for only eight years, from 1864 to 1872. It was quite large at 23 mm — slightly larger than a modern nickel. Right: The three-cent coin was produced for 38 years, from 1851 to 1889. It was small at 14 mm — slightly smaller than a modern dime.


This entry was posted in PyTorch. Bookmark the permalink.

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s