Category Archives: Machine Learning

Another Look at Neural Network Dropout

I spent a few hours taking a fresh look at a technique called neural network dropout. Dropout is a relatively simple technique (in principle anyway) that is used when training a neural network, and is intended to prevent model overfitting. … Continue reading

Posted in Machine Learning | Leave a comment

I do an Interview with InformationWeek Magazine

A few weeks ago I spoke at the 2017 Interop Conference. Interop is a very large conference with many topics related to IT in general — hardware, software, management, and so on. My talk at Interop was an introduction to … Continue reading

Posted in Conferences, Machine Learning

A Neural Network Mini-Batcher

When training a neural network, you can update weights after reading each training data item (usually called online training) or you can update weights after reading all data items (usually called batch training). Both types of training have their pros … Continue reading

Posted in Machine Learning

Neural Network Saturation

A neural network classifier has a set of input nodes, a set of hidden processing nodes, and a set of output nodes. The hidden nodes usually have values between -1.0 and +1.0 if you use the tanh activation function. The … Continue reading

Posted in Machine Learning

Introduction to the Microsoft CNTK v2.0 Library

I wrote an article titled “Introduction to the Microsoft CNTK v2.0 Library” in the July 2017 issue of Microsoft MSDN Magazine. See https://msdn.microsoft.com/en-us/magazine/mt784662. The CNTK library is a powerful set of functions that allow a developer to create deep neural … Continue reading

Posted in CNTK, Machine Learning | 1 Comment

Neural Network Error: Output Minus Target or Target Minus Output

The neural network back-propagation algorithm has many subtle details. If you assume a squared error function with squared output minus target, the weight update rule adds the delta-weight. If you assume target minus output, the weight update rule subtracts the … Continue reading

Posted in Machine Learning | 3 Comments

A Neural Network Equivalent to Logistic Regression

When I was first studying machine learning, I sometimes wondered about the relationship between logistic regression and neural networks. When I did an Internet search on the topic recently, I saw all kinds of rather confusing information. Logistic regression is … Continue reading

Posted in Machine Learning