Author Archives: jamesdmccaffrey

Serving Up PyTorch Training Data Using The DataLoader collate_fn Parameter

When creating a deep neural network, writing code to prepare the training data and serve it up in batches to the network is almost always difficult and time consuming. A regular PyTorch DataLoader works great for tabular style data where … Continue reading

Posted in PyTorch | Leave a comment

Integrated Gradient Intepretability Explained

One of many techniques to explain why a neural network produced a particular prediction is called integrated gradient. The idea is difficult to understand if you’re not familiar with it. So I’ll try to give an informal (as possible) explanation. … Continue reading

Posted in Machine Learning | Leave a comment

An Example of the Python SciPy line_search() Function

The term “line search” refers to a classical statistics technique to minimize a function. The Python SciPy code library has a line_search() function that is a helper function for a line search but line_search() doesn’t actually do a line search. … Continue reading

Posted in Machine Learning | Leave a comment

Why I’m Not a Fan of the Julia Programming Language

The Julia programming language is a relatively new (still in development) general purpose language intended mostly for use with numerical and scientific programming. I tried out Julia v1.0 about two years ago and wasn’t impressed — at that time Julia … Continue reading

Posted in Machine Learning | Leave a comment

Reducing the Size of a Neural Network using Single-Shot Network Pruning at Initialization

Neural networks can be huge. A neural network with millions or billions of weights and biases (“trainable parameters”) can take weeks to train, which would cost a lot of money, and emit a lot of CO2 from the energy consumption. … Continue reading

Posted in Machine Learning | Leave a comment

Generating Synthetic Data Using a Generative Adversarial Network (GAN) with PyTorch in Visual Studio Magazine

I wrote an article titled “Generating Synthetic Data Using a Generative Adversarial Network (GAN) with PyTorch” in the June 2021 edition of Microsoft Visual Studio Magazine. See https://visualstudiomagazine.com/articles/2021/06/02/gan-pytorch.aspx. A generative adversarial network (GAN) is a deep neural system that can … Continue reading

Posted in PyTorch | Leave a comment

Seven Deep Learning Techniques for Unsupervised Anomaly Detection

The goal of anomaly detection is to examine a set of data to find unusual data items. Three of the main approaches are 1.) rule based techniques, 2.) classification techniques from labeled training data, 3.) unsupervised techniques. Suppose some source … Continue reading

Posted in Machine Learning | Leave a comment

Combining Logistic Regression Models by Averaging Their Weights

Suppose you want to make a logistic regression binary classification model, for example to predict if a hospital patient is male or female based on variables such as age and hospitalization history. And suppose your training data file is very … Continue reading

Posted in PyTorch | Leave a comment

Creating a PyTorch Vocabulary Object for Natural Language Processing Problems

Even the simplest natural language processing problem is extremely difficult. With a basic neural network classifier, you usually have to normalize numeric data (such as dividing a person’s age by 100) and encode non-numeric data (e.g., “red” = 1,0,0) but … Continue reading

Posted in PyTorch | Leave a comment

The Hellinger Distance Between Two Probability Distributions Using Python

A fairly common sub-problem when working with machine learning algorithms is to compute the distance between two probability distributions. For example, suppose distribution P = (0.36, 0.48, 0.16) and Q = (0.33, 0.33, 0.33). What is the difference between P … Continue reading

Posted in Machine Learning | Leave a comment