Category Archives: PyTorch

PyTorch Multi-Class Accuracy By Class Using a Set-Wise Approach

I recently revisited multi-class classification using PyTorch. My demo was to predict a person’s political type (conservative, moderate, liberal) based on sex, age, state (michigan, nebraska, oklahoma), and annual income. See jamesmccaffrey.wordpress.com/2022/09/01/multi-class-classification-using-pytorch-1-12-1-on-windows-10-11/. My demo computed overall model accuracy. I decided … Continue reading

Posted in PyTorch | Leave a comment

Binary Classification Using PyTorch 1.12.1 on Windows 10/11

There are frequent updates to the PyTorch neural network library, and I’m continuously learning new techniques and best practices. I figured it was time to update one of my standard binary classification demos for the current PyTorch version 1.12.1. I … Continue reading

Posted in PyTorch | Leave a comment

PyTorch Not-Fully-Connected Layer Using prune.custom_from_mask()

I ran across an interesting PyTorch function that I hadn’t seen before. The torch.nn.utils.prune.custom_from_mask() function can mask out weights and biases in a neural layer. This allows you to create layers that are not fully connected. I checked the PyTorch … Continue reading

Posted in PyTorch | Leave a comment

Why PyTorch Layer Weight Matrix Shape Seems Backward

A PyTorch weight matrix has shape [num_out, num_in] rather than the more logical [num_in, num_out]. This seems a bit strange. Furthermore, when computing a set of output nodes, the weight matrix must be transposed before applying matrix multiplication. This seems … Continue reading

Posted in PyTorch | Leave a comment

“Multi-Class Classification Using PyTorch, Part 1: New Best Practices” in Visual Studio Magazine

I wrote an article titled “Multi-Class Classification Using PyTorch, Part 1: New Best Practices” in the September 2022 edition of Microsoft Visual Studio Magazine. See https://visualstudiomagazine.com/articles/2022/09/06/multi-class-pytorch.aspx. A multi-class classification problem is one where the goal is to predict a discrete … Continue reading

Posted in PyTorch | Leave a comment

Using the Simplest Possible Transformer Sequence-to-Sequence Example

I’ve been exploring PyTorch Transformer Architecture models sequence-to-sequence problems for several months. TA architecture systems are among the most complicated software things I’ve ever worked with. I recently completed a demo implementation of my idea of the simplest possible sequence-to-sequence. … Continue reading

Posted in PyTorch, Transformers | Leave a comment

The Simplest Possible PyTorch Transformer Sequence-to-Sequence Example

I’ve been looking at PyTorch transformer architecture (TA) networks. TA networks are among the most complex software components I’ve ever worked with, in terms of both conceptual complexity and engineering difficulty. I set out to implement the simplest possible transformer … Continue reading

Posted in PyTorch, Transformers | Leave a comment

Why Does PyTorch Have Three Different Tanh() Functions?

When I was first learning how to use the PyTorch neural network library, I remember being confused by some of the example programs I found on the Internet. There seemed to be three different tanh() functions. After some investigation I … Continue reading

Posted in PyTorch | Leave a comment

Printing PyTorch Model Summary Information

When I use the Keras neural network library, I often use the built-in model.summary() function to display information about a network such as the number of weights and biases. When I use the PyTorch neural network library, I rarely display … Continue reading

Posted in PyTorch | Leave a comment

Multi-Class Classification Using PyTorch 1.12.1 on Windows 10/11

One of the challenges of working with the PyTorch neural network library is that there are constant updates. Over the past two years there have been approximately 16 new releases — roughly a new release every six or seven weeks … Continue reading

Posted in PyTorch | Leave a comment