Deep neural networks and deep learning are hot topics now — machine learning goes through fads and fashions just like everything else. There’s no clear consensus on exactly what deep neural networks are or what deep learning means. But from a practical point of view, a deep neural network is one with two or more layers of hidden nodes.

I wrote an article titled “Deep Neural Networks: A Getting Started Tutorial” in the June 2014 issue of Visual Studio Magazine. See http://visualstudiomagazine.com/articles/2014/06/01/deep-neural-networks.aspx. In the article I present a demo program, written in C#, of a neural network with two layers of hidden nodes.

Deep neural networks can, in theory, solve some prediction problems that ordinary neural networks cannot. Also, deep neural networks can, again in theory, solve some prediction problems in situations where an ordinary NN would require a huge number of hidden nodes in its single hidden layer.

The idea of deep neural networks is not new, but what has changed is that the computing power to implement deep neural networks is now starting to become available. My article shows the basics of a two-hidden-layer neural network but does not discuss how to train the beast — that’s for a future article.

### Like this:

Like Loading...

*Related*

The Windows neural network application that I first developed in 1997 uses multiple hidden layers of nodes so I supposed it’s “deep”. I find in practice that most problems only need a single hidden layer and no problems needs more than two.

Interesting. I work with a bunch of guys who are serious experts. I’ll ask around to see if anyone finds use for neural networks with more than two hidden layers.

I talked with John Platt today. He’s a famous ML researcher who works at Microsoft. He knows ML. John noted that in the vast majority of situations, a single layer of hidden nodes is all that is needed (Cybenko Theorem). But some specific state-of-the-art applications, like speech recognition, use many (like 10 – 20 or more) hidden layers.