Deep neural networks and deep learning are hot topics now — machine learning goes through fads and fashions just like everything else. There’s no clear consensus on exactly what deep neural networks are or what deep learning means. But from a practical point of view, a deep neural network is one with two or more layers of hidden nodes.
I wrote an article titled “Deep Neural Networks: A Getting Started Tutorial” in the June 2014 issue of Visual Studio Magazine. See http://visualstudiomagazine.com/articles/2014/06/01/deep-neural-networks.aspx. In the article I present a demo program, written in C#, of a neural network with two layers of hidden nodes.
Deep neural networks can, in theory, solve some prediction problems that ordinary neural networks cannot. Also, deep neural networks can, again in theory, solve some prediction problems in situations where an ordinary NN would require a huge number of hidden nodes in its single hidden layer.
The idea of deep neural networks is not new, but what has changed is that the computing power to implement deep neural networks is now starting to become available. My article shows the basics of a two-hidden-layer neural network but does not discuss how to train the beast — that’s for a future article.