Category Archives: PyTorch

Computing log_softmax() for PyTorch Directly

In a PyTorch multi-class classification problem, the basic architecture is to apply log_softmax() activation on the output nodes, in conjunction with NLLLoss() during training. It’s possible to compute softmax() and then apply log() but it’s slightly more efficient to compute … Continue reading

Posted in PyTorch | Leave a comment

Custom Loss Functions for PyTorch

The PyTorch neural network code library has built-in loss functions that can handle most scenarios. Examples include NLLLoss() and CrossEntropyLoss() for multi-class classification, BCELoss() for binary classification, and MSELoss() and L1Loss() for regression. Because PyTorch works at a low level … Continue reading

Posted in PyTorch | Leave a comment

Multi-Class Classification Using PyTorch 1.12.1-CPU on MacOS

I do most of my work on Windows OS machines. One morning I noticed that my MacBook laptop in my office was collecting dust so I figured I’d upgrade the existing PyTorch 1.10.0 to version 1.12.1 to make sure there … Continue reading

Posted in PyTorch | Leave a comment

“Regression Using PyTorch, Part 1: New Best Practices” in Visual Studio Magazine

I wrote an article titled “Regression Using PyTorch, Part 1: New Best Practices” in the November 2022 edition of Microsoft Visual Studio Magazine. See https://visualstudiomagazine.com/articles/2022/11/01/pytorch-regression.aspx. A regression problem is one where the goal is to predict a single numeric value. … Continue reading

Posted in PyTorch | Leave a comment

Analyzing PyTorch Using the Bottleneck Utility

I hadn’t used the torch.utils.bottleneck tool for quite some time, so just for hoots I figured I’d see if anything had changed since my last use of the tool. The bottleneck tool analyzes a program run using the Python profiler … Continue reading

Posted in PyTorch | Leave a comment

PyTorch Model Training Using a Program-Defined Function vs. Inline Code

In most cases, I train a PyTorch model using inline code inside a main() function. For example: import torch as T . . . def main(): # 0. get started print(“Begin People predict politics type “) T.manual_seed(1) np.random.seed(1) # 1. … Continue reading

Posted in PyTorch | 1 Comment

An Example of PyTorch Hyperparameter Random Search

Bottom line: Hyperparameter random search can be effective but the difficult part is determining what to parameterize and the range of possible parameter values. When creating a neural network prediction model there are many architecture hyperparameters (number hidden layers, number … Continue reading

Posted in PyTorch | Leave a comment

Yet Another Implementation of a softargmax() Function for PyTorch

Suppose you have a vector/tensor t = [5.5, 2.2, 7.7, 9.9, 8.8]. The regular argmax(t) gives the index of the largest value in the tensor. Because the largest value is 9.9 at [3], argmax(t) = 3. In some rare scenarios, … Continue reading

Posted in PyTorch | 1 Comment

“Binary Classification Using New PyTorch Best Practices, Part 2: Training, Accuracy, Predictions” in Visual Studio Magazine.

I wrote an article titled “Binary Classification Using New PyTorch Best Practices, Part 2: Training, Accuracy, Predictions” in the October 2022 edition of Microsoft Visual Studio Magazine. See https://visualstudiomagazine.com/articles/2022/10/14/binary-classification-using-pytorch-2.aspx. The goal of a binary classification system is to predict the … Continue reading

Posted in PyTorch | Leave a comment

Transformer Based Dataset Similarity for MNIST

Computing a measure of the similarity of two datasets is a very difficult problem. Several months ago, I worked on a project and devised an algorithm based on a neural autoencoder. See jamesmccaffrey.wordpress.com/2021/04/02/dataset-similarity-sometimes-ideas-work-better-than-expected/. I wondered if I could adapt my … Continue reading

Posted in PyTorch, Transformers | Leave a comment