Author Archives: jamesdmccaffrey

The Simplest Possible PyTorch Transformer Sequence-to-Sequence Example

I’ve been looking at PyTorch transformer architecture (TA) networks. TA networks are among the most complex software components I’ve ever worked with, in terms of both conceptual complexity and engineering difficulty. I set out to implement the simplest possible transformer … Continue reading

Posted in PyTorch, Transformers | Leave a comment

NFL 2022 Week 1 Predictions – Zoltar Likes Seven Underdogs

Zoltar is my NFL football prediction computer program. It uses reinforcement learning and a neural network. Here are Zoltar’s predictions for week #1 of the 2022 season. These predictions are tentative, in the sense that it usually takes Zoltar about … Continue reading

Posted in Zoltar | Leave a comment

Why Does PyTorch Have Three Different Tanh() Functions?

When I was first learning how to use the PyTorch neural network library, I remember being confused by some of the example programs I found on the Internet. There seemed to be three different tanh() functions. After some investigation I … Continue reading

Posted in PyTorch | Leave a comment

Printing PyTorch Model Summary Information

When I use the Keras neural network library, I often use the built-in model.summary() function to display information about a network such as the number of weights and biases. When I use the PyTorch neural network library, I rarely display … Continue reading

Posted in PyTorch | Leave a comment

The Maia Computer Chess Program

I came across an interesting research project named Maia. Maia is a computer chess program that was designed to play more like a human than conventional chess programs. See There is a long and fascinating history of chess, computer … Continue reading

Posted in Machine Learning | Leave a comment

Multi-Class Classification Using PyTorch 1.12.1 on Windows 10/11

One of the challenges of working with the PyTorch neural network library is that there are constant updates. Over the past two years there have been approximately 16 new releases — roughly a new release every six or seven weeks … Continue reading

Posted in PyTorch | Leave a comment

An Example of Warm-Start Training for PyTorch

When you train a neural network from scratch, the weights and biases of the network are initialized with random values. In warm-start training, instead of using random values for initialization, you use the weights and biases of a different trained … Continue reading

Posted in PyTorch | Leave a comment

A Quick Look at PyTorch Audio

A few days ago, a work colleague asked me for some advice on machine learning with audio. My quick reply was that I know very little about working with audio data. So, I pointed him to some other ML experts. … Continue reading

Posted in PyTorch | Leave a comment

Fashion-MNIST Classification Example Using PyTorch

The MNIST (Modified National Institute of Standards and Technology) dataset has 60,000 training and 10,000 test data items. Each item is a crude image of a handwritten digit from ‘0’ to ‘9’. Each image is 28 by 28 pixels, and … Continue reading

Posted in PyTorch | Leave a comment

PyTorch Custom Weight Initialization Example

Neural network weight and bias initialization is a surprisingly tricky topic. In most cases, the default initialization code works fine, even though it’s poorly documented and the source code is extremely complex. Just to test my understanding of weight and … Continue reading

Posted in PyTorch | Leave a comment