“Multi-Class Classification Using PyTorch, Part 1: New Best Practices” in Visual Studio Magazine

I wrote an article titled “Multi-Class Classification Using PyTorch, Part 1: New Best Practices” in the September 2022 edition of Microsoft Visual Studio Magazine. See https://visualstudiomagazine.com/articles/2022/09/06/multi-class-pytorch.aspx.

A multi-class classification problem is one where the goal is to predict a discrete value where there are three or more possibilities. For example, you might want to predict the political leaning (conservative, moderate, liberal) of a person based on their sex, age, state where they live and annual income.

I’d written previous articles on multi-class classification, but machine learning with deep neural techniques has advanced quickly. The article updates multi-class classification techniques and best practices based on experience I’d gained over the past two years.

The article explains a demo program. The demo begins by loading a 200-item file of training data and a 40-item set of test data. Each tab-delimited line represents a person. The fields are sex, age, state of residence, annual income and politics type (0 = conservative, 1 = moderate and 2 = liberal). The goal is to predict politics type from sex, age, state and income.

After the training data is loaded into memory, the demo creates a 6-(10-10)-3 neural network. This means there are six input nodes, two hidden neural layers with 10 nodes each and three output nodes.

The demo prepares to train the network by setting a batch size of 10, stochastic gradient descent (SGD) optimization with a learning rate of 0.01 and maximum training epochs of 1,000 passes through the training data.

The demo program monitors training by computing and displaying the loss value for one epoch. The loss value slowly decreases, which indicates that training is probably succeeding. The magnitude of the loss values isn’t directly interpretable; the important thing is that the loss decreases.

After 1,000 training epochs, the demo program computes the accuracy of the trained model on the training data as 81.50 percent (163 out of 200 correct). The model accuracy on the test data is 75.00 percent (30 out of 40 correct).

After evaluating the trained network, the demo predicts the politics type for a person who is male, 30 years old, from Oklahoma, who makes $50,000 annually. The prediction is [0.6905, 0.3049, 0.0047]. These values are pseudo-probabilities. The largest value (0.6905) is at index [0] so the prediction is class 0 = conservative.

The demo concludes by saving the trained model to file so that it can be used without having to retrain the network from scratch. There are two different ways to save a PyTorch model. The demo uses the save-state approach.

The article explains the first parts of the demo: installing PyTorch, creating the data files and Dataset object, and defining the neural network. A second article will explain how to train the network, evaluate network accuracy, use a trained network, and save a network.



I lived in Hawaii from 1984 to 1997. Left: Style classification = 1950s mid-century modern. Center: Style classification = 1930s art deco. Right: Style classification = 1960s psychedelic.


This entry was posted in PyTorch. Bookmark the permalink.

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s