Author Archives: jamesdmccaffrey

When I was first learning how to use the PyTorch neural network library, I remember being confused by some of the example programs I found on the Internet. There seemed to be three different tanh() functions. After some investigation I … Continue reading →

When I use the Keras neural network library, I often use the built-in model.summary() function to display information about a network such as the number of weights and biases. When I use the PyTorch neural network library, I rarely display … Continue reading →

One of the challenges of working with the PyTorch neural network library is that there are constant updates. Over the past two years there have been approximately 16 new releases — roughly a new release every six or seven weeks … Continue reading →

When you train a neural network from scratch, the weights and biases of the network are initialized with random values. In warm-start training, instead of using random values for initialization, you use the weights and biases of a different trained … Continue reading →

A few days ago, a work colleague asked me for some advice on machine learning with audio. My quick reply was that I know very little about working with audio data. So, I pointed him to some other ML experts. … Continue reading →

The MNIST (Modified National Institute of Standards and Technology) dataset has 60,000 training and 10,000 test data items. Each item is a crude image of a handwritten digit from ‘0’ to ‘9’. Each image is 28 by 28 pixels, and … Continue reading →

Neural network weight and bias initialization is a surprisingly tricky topic. In most cases, the default initialization code works fine, even though it’s poorly documented and the source code is extremely complex. Just to test my understanding of weight and … Continue reading →