Author Archives: jamesdmccaffrey

Computing log_softmax() for PyTorch Directly

In a PyTorch multi-class classification problem, the basic architecture is to apply log_softmax() activation on the output nodes, in conjunction with NLLLoss() during training. It’s possible to compute softmax() and then apply log() but it’s slightly more efficient to compute … Continue reading

Posted in PyTorch | Leave a comment

Why Neural Network Training Momentum Isn’t Used Very Often

During neural network training, it’s possible to use a momentum factor. Momentum is a technique designed to speed up training. But I hardly ever see momentum used. The main problem with momentum is that it adds another hyperparameter, the momentum … Continue reading

Posted in Machine Learning | 1 Comment

NFL 2022 Week 13 Predictions – Zoltar Likes the Raiders to Beat the Chargers

Zoltar is my NFL football prediction computer program. It uses reinforcement learning and a neural network. Here are Zoltar’s predictions for week #13 of the 2022 season. Zoltar: patriots by 1 dog = bills Vegas: bills by 5 Zoltar: steelers … Continue reading

Posted in Zoltar | Leave a comment

Mahalanobis Distance Example Using Python

Suppose you have a source dataset of five items where each item is a person’s height, test-score, age: [64.0, 580.0, 29.0] [66.0, 570.0, 33.0] [68.0, 590.0, 37.0] [69.0, 660.0, 46.0] [73.0, 600.0, 55.0] And suppose you want the Mahalanobis distance … Continue reading

Posted in Machine Learning | Leave a comment

Custom Loss Functions for PyTorch

The PyTorch neural network code library has built-in loss functions that can handle most scenarios. Examples include NLLLoss() and CrossEntropyLoss() for multi-class classification, BCELoss() for binary classification, and MSELoss() and L1Loss() for regression. Because PyTorch works at a low level … Continue reading

Posted in PyTorch | Leave a comment

“Researchers Explore Machine Learning Hyperparameter Tuning Using Evolutionary Optimization” on the Pure AI Web Site

I contributed to an article titled “Researchers Explore Machine Learning Hyperparameter Tuning Using Evolutionary Optimization” in the November 2022 edition of the Pure AI web site. See https://pureai.com/articles/2022/11/01/evolutionary-optimization.aspx. When data scientists create a machine learning prediction model, there are typically … Continue reading

Posted in Machine Learning | Leave a comment

NFL 2022 Week 12 Predictions – Zoltar Likes Seven Underdogs

Zoltar is my NFL football prediction computer program. It uses reinforcement learning and a neural network. Here are Zoltar’s predictions for week #12 of the 2022 season. Zoltar: bills by 1 dog = lions Vegas: bills by 10 Zoltar: cowboys … Continue reading

Posted in Zoltar | Leave a comment

Recap of the Fall 2022 MLADS Conference

I gave a technical talk titled “Simple Unsupervised Anomaly Detection Using a PyTorch Transformer Autoencoder” at the Fall 2022 Machine Learning Artificial Intelligence and Data Science (MLADS) conference. The MLADS conference is an internal event at the large tech company … Continue reading

Posted in Conferences | Leave a comment

Simple Numerical Optimization Using an Evolutionary Algorithm with C#

The goal of a numerical optimization problem is to find a vector of values that minimize some cost function. The most fundamental example is minimizing the Sphere Function f(x0, x1, .. xn) = x0^2 + x1^2 + .. + xn^2. … Continue reading

Posted in Machine Learning | Leave a comment

Multi-Class Classification Using PyTorch 1.12.1-CPU on MacOS

I do most of my work on Windows OS machines. One morning I noticed that my MacBook laptop in my office was collecting dust so I figured I’d upgrade the existing PyTorch 1.10.0 to version 1.12.1 to make sure there … Continue reading

Posted in PyTorch | Leave a comment