Category Archives: Machine Learning

Researchers Use Machine Learning Techniques to Detect Compromised Network Accounts on the Pure AI

I contributed to an article titled “Researchers Use Machine Learning Techniques to Detect Compromised Network Accounts” on the Pure AI web site. See https://pureai.com/articles/2021/07/06/ml-detect.aspx. The article describes how researchers and engineers (including me) developed a successful system that detects compromised … Continue reading

Posted in Machine Learning | Leave a comment

Differential Evolution Optimization Example Using Python

An Evolutionary Algorithm (EA) is one of many algorithms that are loosely based on the biological ideas of genetic crossover and mutation. Differential Evolution (DE) is a specific type of EA that has a bit of structure. I’m very familiar … Continue reading

Posted in Machine Learning | Leave a comment

Computing PCA Using NumPy Without Scikit

Principal component analysis (PCA) is a classical statistics technique that can do data dimensionality reduction. This can be used to graph high dimensional data (if you reduce the dim to 2), or to clean data (by reconstructing the data from … Continue reading

Posted in Machine Learning, PyTorch | 1 Comment

Particle Swarm Optimization Variants

Particle swarm optimization (PSO) is a meta-heuristic that can be used to construct a specific algorithm to find the minimum of an error function. In theory, PSO could improve neural network training because PSO does not use Calculus gradients like … Continue reading

Posted in Machine Learning | Leave a comment

Sentiment Analysis Using a PyTorch EmbeddingBag Layer in Visual Studio Magazine

I wrote an article titled “Sentiment Analysis Using a PyTorch EmbeddingBag Layer” in the July 2021 edition of the online Microsoft Visual Studio Magazine. See https://visualstudiomagazine.com/articles/2021/07/06/sentiment-analysis.aspx. Natural language processing (NLP) problems are very difficult. A common type of NLP problem … Continue reading

Posted in Machine Learning, PyTorch | Leave a comment

Spiral Dynamics Inspired Optimization Demo Using Python

I’ve been looking at an interesting optimization algorithm based on a 2011 research paper titled “Spiral Dynamics Inspired Optimization” by K. Tamura and K. Yasuda. SDI optimization is somewhat similar to particle swarm optimization (SPO). Briefly, SDI sets up a … Continue reading

Posted in Machine Learning | Leave a comment

Rastrigin Function Graph Using matplotlib With Edge Colors

The Rastrigin function is a standard benchmark problem for optimization algorithms. The general form of the function in n dimensions is f(X) = Sum[xi^2 – 10*cos(2*pi*xi)] + 10n. For the specific case when n = 2, the function is f(x1, … Continue reading

Posted in Machine Learning | Leave a comment

Anomaly Detection Using Principal Component Analysis (PCA) Reconstruction Error

I was working on an anomaly detection system recently. The system used a deep neural autoencoder. As part of the system evaluation, we looked at anomaly detection using principal component analysis (PCA). PCA is a classical statistics technique that decomposes … Continue reading

Posted in Machine Learning | Leave a comment

Spiral Dynamics Optimization in 3D

I’m slowly but surely dissecting the ideas in a 2011 research paper titled “Spiral Dynamics Inspired Optimization” by K. Tamura and K. Yasuda. The idea of SDI optimization is an algorithm to find the minimum value of some function using … Continue reading

Posted in Machine Learning | Leave a comment

Why Do I Never Remember the Differences Between ROC AUC and PR AUC?

I was working on an anomaly detection system recently. Whem working with any machine learning prediction system, you should evaluate the effectiveness of the system. The basic effectiveness metric is prediction accuracy. But in systems where there is imbalanced data, … Continue reading

Posted in Machine Learning | Leave a comment