Author Archives: jamesdmccaffrey

Two Ways to Deal with the Derivative of the ReLU Function

I gave a talk about the back-propagation algorithm recently. Back-propagation is used to train a neural network. Consider a math equation like y = 5X1 + 7X2 so the equation has two inputs, X1 and X2, and two constants, 5 … Continue reading

Posted in Machine Learning | Leave a comment

I Give a Talk “Introduction to Deep Neural Networks”

I gave a talk titled “Introduction to Deep Neural Networks” recently. The goal was to give the audience of engineers the information they needed to understand what types of problems can be solved using a DNN, and what tools and … Continue reading

Posted in Machine Learning, Conferences | 2 Comments

Neural Network Glorot Initialization

You’d think that initializing the weights and biases in a neural network wouldn’t be very difficult or interesting. No so. The simplest way to initialize weights and biases is to set them to small (perhaps -0.01 to +0.01) uniform random … Continue reading

Posted in Machine Learning | Leave a comment

Neural Network Back-Propagation using Python

I wrote an article titled “Neural Network Back-Propagation using Python” in the June 2017 issue of Visual Studio Magazine. See I strongly believe that when working with machine learning, even if you’re using a tool such as Weka or … Continue reading

Posted in Machine Learning | 1 Comment and Microsoft Talk Sports Technology ( is a Seattle-based Web site and is quite well-known in the Pacific Northwest. The site posts all kinds of interesting, tech-related stories. I had read a few sports-tech related articles written by a reporter, Taylor Soper. So, … Continue reading

Posted in Conferences, Miscellaneous | Leave a comment

Graphing Rastrigin’s Function using the Matplotlib Library

I thought I’d refresh my memory of the matplotlib library, an add-on package for Python that can create plots and graphs. I regularly use the plotting functions in R, SciLab, Excel, and Python, but using these isn’t easy or intuitive … Continue reading

Posted in CNTK | Leave a comment

The Greatest Chess Tournament of All Time

I haven’t played an actual game of chess in many years, but I used to love to play chess in high school, when I had a lot of time. I still follow chess, mostly via the excellent In June … Continue reading

Posted in Miscellaneous, Top Ten | 2 Comments