Some Beautiful Neural Network Visualizations by Thorsten Kleppe

A fellow machine learning enthusiast, Thorsten Kleppe, sent me an email message that pointed me to a GitHub project he owns. The project had some very beautiful animated visualizations of neural networks for the MNIST image data set. See https://github.com/grensen/gif_test/blob/master/README.md.

The MNIST data set has 70,000 images of hand-drawn digits from ‘0’ to ‘9’. Each digit is 28 x 28 pixels. The goal is to classify an image based on the 784 pixel values.

---

This is a screenshot capture of an animation of how Thorsten’s system classifies a ‘3’ and identifies unneeded nodes.

---

The system can interactively prune unneeded nodes – notice the mouse cursor pointing to a node.

---

This screenshot shows the system control panel.

---

In addition to the animated visualizations, Thorsten used a novel approach to optimize training. I didn’t have time to look at all the details, but the gist of the idea is to prune away unneeded nodes.

Neural network visualizations are one of the best ways to tackle the problem of neural network interpretability. A visualization doesn’t reveal exactly what’s going on, but a visualization can provide good insights.

Very nice work.

---

Vogue Magazine is widely considered (according to the Internet at least) to be an influential channel for visualizations of beauty. Left: Cover from September 1957. Center: Cover from July 1935. Right: Cover from April 1963. Of the three, I like the center image best — I tend to prefer an abstraction of beauty to a reality of beauty.