Basic Deep Neural Network Input-Output

There’s been a great increase in interest recently on the topic of deep neural networks (DNNs). The term DNN is general and somewhat ambiguous. A regular neural network has one layer of input nodes, one layer of hidden processing nodes, and one layer of output nodes. Deep neural networks have multiple hidden layers of processing nodes.

There are specific kinds of DNNs that have complicated architectures. They have names like convolutional neural networks (CNNs), recurrent neural networks (RNNs), residual neural networks (ResNets), and many more. In order to understand these exotic DNNs, in my opinion it’s necessary to have a solid understanding of basic DNNs — neural networks with multiple hidden layers but nothing fancy with regards to connection architecture.

So I spent a few hours coding up basic deep neural network from scratch, using the C# language. It wasn’t too difficult, but it was time-consuming, and there were many, many different design choices.

My demo calling code is:

Console.WriteLine("\nBegin deep net IO demo \n");

Console.WriteLine("Creating a 2-(4-2-2)-3 BDNN ");
int numInput = 2;
int[] numHidden = new int[] { 4, 2, 2 };
int numOutput = 3;
DeepNet dn = new DeepNet(numInput, numHidden, numOutput);

int nw = DeepNet.NumWeights(numInput, numHidden, numOutput);
Console.WriteLine("Setting weights and biases to 0.01 to " +
  (nw/100.0).ToString("F2") );
double[] wts = new double[nw];
for (int i = 0; i < wts.Length; ++i)
  wts[i] = (i + 1) * 0.01;  // [0.01 . . 0.37]
dn.SetWeights(wts);  // weights + biases

Console.WriteLine("\nComputing output values for input" +
  " = [11.0, 22.0] \n");
double[] xValues = new double[] { 11.0, 22.0 };
dn.Dump(true);  // show weights

Console.WriteLine("\nEnd demo \n");

Well I’ve got the input-output mechanism working. Next, I’ll tackle back-propagation training for a basic DNN. That will be more difficult I’m sure.

This entry was posted in Machine Learning. Bookmark the permalink.

2 Responses to Basic Deep Neural Network Input-Output

  1. will wang says:

    james, any chance to check the source code of the DeepNet class

Comments are closed.