Category Archives: Machine Learning

Integrated Gradient Intepretability Explained

One of many techniques to explain why a neural network produced a particular prediction is called integrated gradient. The idea is difficult to understand if you’re not familiar with it. So I’ll try to give an informal (as possible) explanation. … Continue reading

Posted in Machine Learning | Leave a comment

An Example of the Python SciPy line_search() Function

The term “line search” refers to a classical statistics technique to minimize a function. The Python SciPy code library has a line_search() function that is a helper function for a line search but line_search() doesn’t actually do a line search. … Continue reading

Posted in Machine Learning | Leave a comment

Why I’m Not a Fan of the Julia Programming Language

The Julia programming language is a relatively new (still in development) general purpose language intended mostly for use with numerical and scientific programming. I tried out Julia v1.0 about two years ago and wasn’t impressed — at that time Julia … Continue reading

Posted in Machine Learning | Leave a comment

Reducing the Size of a Neural Network using Single-Shot Network Pruning at Initialization

Neural networks can be huge. A neural network with millions or billions of weights and biases (“trainable parameters”) can take weeks to train, which would cost a lot of money, and emit a lot of CO2 from the energy consumption. … Continue reading

Posted in Machine Learning | Leave a comment

Seven Deep Learning Techniques for Unsupervised Anomaly Detection

The goal of anomaly detection is to examine a set of data to find unusual data items. Three of the main approaches are 1.) rule based techniques, 2.) classification techniques from labeled training data, 3.) unsupervised techniques. Suppose some source … Continue reading

Posted in Machine Learning | Leave a comment

The Hellinger Distance Between Two Probability Distributions Using Python

A fairly common sub-problem when working with machine learning algorithms is to compute the distance between two probability distributions. For example, suppose distribution P = (0.36, 0.48, 0.16) and Q = (0.33, 0.33, 0.33). What is the difference between P … Continue reading

Posted in Machine Learning | Leave a comment

The Worst Logistic Regression Graph Diagram on the Internet

Argh! I have to post on this topic. Strewn throughout the Internet is a graph that is supposed to explain what logistic regression is and how it works. I’ve seen this graph, and variations of it, for years and it … Continue reading

Posted in Machine Learning | 2 Comments

Neural Network Lottery Ticket Hypothesis: The Engineer In Me Is Not Impressed

The neural network lottery ticket hypothesis was proposed in a 2019 research paper titled “The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks” by J. Frankle and M. Carbin. Their summary of the idea is: We find that a standard … Continue reading

Posted in Machine Learning | Leave a comment

Implementing Kullback-Leibler Divergence from Scratch Using Python

The Kullback-Leibler divergence is a number that is a measure of the difference between two probability distributions. I wrote some machine learning code for work recently and I used a version of a KL function from the Python scipy.stats.entropy code … Continue reading

Posted in Machine Learning | Leave a comment

Positive and Unlabeled Learning (PUL) Using PyTorch

I wrote an article titled “Positive and Unlabeled Learning (PUL) Using PyTorch” in the May 2021 edition of the online Microsoft Visual Studio Magazine. See https://visualstudiomagazine.com/articles/2021/05/20/pul-pytorch.aspx. A positive and unlabeled learning (PUL) problem occurs when a machine learning set of … Continue reading

Posted in Machine Learning, PyTorch | Leave a comment