Category Archives: Machine Learning

Kernel Perceptrons using C#

I wrote an article titled “Kernel Perceptrons using C#” in the April 2017 issue of MSDN Magazine. See https://msdn.microsoft.com/en-us/magazine/mt797653. A kernel perceptron is a machine learning technique that can be used to make a binary prediction — one where the … Continue reading

Posted in Machine Learning

The Difference Between Log Loss and Cross Entropy Error

If you’re new to neural networks, you’ll see the terms “log loss” and “cross entropy error” used a lot. Both terms mean the same thing. Multiple, different terms for the same thing is unfortunately quite common in machined learning (ML). … Continue reading

Posted in Machine Learning

The Cosine Similarity of Two Sentences

I was listening to an interesting lecture on Natural Language Processing (NLP) recently. The talk mentioned the cosine similarity of two sentences. I hadn’t used cosine similarity in a long time so I thought I’d refresh my memory. In general, … Continue reading

Posted in Machine Learning

Yet Another Ignorant Misuse of Statistics

Repeat after me: correlation does not mean causation. A recent headline from the Chicago Tribune caught my eye, “Chicago Area Pays Steep Price for Segregation Study Finds”. Briefly, the article points out that Chicago is highly segregated, but that less … Continue reading

Posted in Machine Learning

Basic Deep Neural Network Input-Output

There’s been a great increase in interest recently on the topic of deep neural networks (DNNs). The term DNN is general and somewhat ambiguous. A regular neural network has one layer of input nodes, one layer of hidden processing nodes, … Continue reading

Posted in Machine Learning | 2 Comments

Experimenting with Neural Network L2 Regularization

Regularization is a standard technique used in neural network training. The most common form of regularization is called L2. The idea is to add the sum of squared (the “2” in “L2”) weight values to the error term during training. … Continue reading

Posted in Machine Learning

Experimenting with Neural Network Dropout

Whenever I read about some sort of technology, no matter how clear the explanation is, I never feel that I fully understand the topic unless I can code a demo program. This is probably a character strength and weakness of … Continue reading

Posted in Machine Learning