Category Archives: Machine Learning

A Quick Demo of the DBSCAN Clustering Algorithm

I was reading a research paper this morning and the paper used the DBSCAN (“density-based spatial clustering of applications with noise”) clustering algorithm. DBSCAN is somewhat similar to k-means clustering. Both work only with strictly numeric data. In k-means you … Continue reading

Posted in Machine Learning | Leave a comment

Differential Evolution Optimization in Visual Studio Magazine

I wrote an article titled “Differential Evolution Optimization” in the September 2021 edition of the Microsoft Visual Studio Magazine. See https://visualstudiomagazine.com/articles/2021/09/07/differential-evolution-optimization.aspx. The most common type of optimization for neural network training is some form of stochastic gradient descent (SGD). SGD … Continue reading

Posted in Machine Learning | Leave a comment

Example of Computing Kullback-Leibler Divergence for Continuous Distributions

In this post, I present an example of estimating the Kullback-Leibler (KL) divergence between two continuous distributions using the Monte Carlo technique. Whoa! Just stating the problem has a massive amount of information. The KL divergence is the key part … Continue reading

Posted in Machine Learning, PyTorch | Leave a comment

The Wasserstein Distance Using C#

The Wasserstein distance has many different variations. In its simplest form the Wasserstein distance function measures the distance between two discrete probability distributions For example, if: double[] P = new double[] { 0.6, 0.1, 0.1, 0.1, 0.1 }; double[] Q1 … Continue reading

Posted in Machine Learning | 1 Comment

Another Set of Beautiful Machine Learning Visualizations from Thorsten Kleppe

Thorsten Kleppe is a fellow machine learning enthusiast who creates beautiful ML visualizations. Thorsten sent me some of his latest work. Thorsten’s new visualizations are based on a logistic regression model applied to the MNIST dataset. The MNIST dataset contains … Continue reading

Posted in Machine Learning | 1 Comment

Wasserstein Distance Using C# and Python in Visual Studio Magazine

I wrote an article titled “Wasserstein Distance Using C# and Python” in the August 2021 edition of Microsoft Visual Studio Magazine. See https://visualstudiomagazine.com/articles/2021/08/16/wasserstein-distance.aspx. There are many different ways to measure the distance between two probability distributions. Some of the most … Continue reading

Posted in Machine Learning | Leave a comment

Comparing Wasserstein Distance with Kullback-Leibler Distance

There are many ways to calculate the distance between two probability distributions. Four of the most common are Kullback-Leibler (KL), Jensen-Shannon (JS), Hellinger (H), and Wasserstein (W). When I was in school, I learned that W was superior to KL, … Continue reading

Posted in Machine Learning | Leave a comment

Preparing the Boston Housing Dataset for PyTorch

The Boston Housing dataset is a standard benchmark for regression algorithms. The goal of the Boston Housing problem is to predict the median price of a house in one of 506 towns near Boston. The data has 14 columns. There … Continue reading

Posted in Machine Learning | Leave a comment

Researchers Explore Differential Evolution Optimization for Machine Learning on Pure AI

I contributed to an article titled “Researchers Explore Differential Evolution Optimization for Machine Learning” posted on the Pure AI web site. See https://pureai.com/articles/2021/08/02/differential-evolution-optimization.aspx. The article explains what differential evolution optimization (DEO) is, and describes how researchers at Microsoft demonstrated that … Continue reading

Posted in Machine Learning | Leave a comment

Why I Don’t Use Min-Max or Z-Score Normalization For Neural Networks

Normalization is the process of scaling numeric predictor values so that they’re all roughly in the same range, typically 0.0 to 1.0 (min-max normalization) or about -4.0 to +4.0 (z-score normalization). Over the past few years I have become quite … Continue reading

Posted in Machine Learning | 2 Comments