Revisiting My Neural Network Regression System with Raw JavaScript

A couple of years ago I implemented a neural network regression system (predict a single numeric value) in raw JavaScript. I enjoy coding, even in raw JavaScript, so one Saturday evening I figured I’d revise my old example.

I didn’t run into any major problems but working with raw JavaScript is always a bit slow. My raw JavaScript neural network can only handle a single hidden layer, but even so the results were pretty good.

I used one of my standard regression problem datasets. The data looks like:

1  0.24  1  0  0  0.2950  0  0  1
0  0.39  0  0  1  0.5120  0  1  0
1  0.63  0  1  0  0.7580  1  0  0
. . .

Each line represents a person. The fields are sex (male = 0, female = 1), age (divided by 100), state (Michigan = 100, Nebraska = 010, Oklahoma = 001), income (divided by $100,000) and political leaning (conservative = 100, moderate = 010, liberal = 001). The goal is to predict income from sex, age, state, and politics.

Implementing a neural network from scratch (in any language) is difficult. My implementation is several hundred lines of code long so I can’t present it in its entirety in this blog post.

Loading the training and test data looks like:

let U = require("../../Utilities/utilities_lib.js");
let FS = require("fs");

function main()
{
  console.log("Begin binary classification demo  ");

  //  1  0.24  1  0  0  0.2950  0  0  1
  // -1  0.39  0  0  1  0.5120  0  1  0

  // 1. load data
  let trainX = U.loadTxt3(".\\Data\\people_train.txt",
    "\t", [0,1,2,3,4,6,7,8], "//");
  let trainY = U.loadTxt3(".\\Data\\people_train.txt",
    "\t", [5], "//");
  let testX = U.loadTxt3(".\\Data\\people_test.txt",
    "\t", [0,1,2,3,4,6,7,8], "//");
  let testY = U.loadTxt3(".\\Data\\people_test.txt",
    "\t", [5], "//");
. . .

And creating and training the network is:

  // 2. create network
  console.log("\nCreating 8-100-1 tanh, Identity NN ");
  let seed = 0;
  let nn = new NeuralNet(8, 100, 1, seed);

  // 3. train network
  let lrnRate = 0.005;
  let maxEpochs = 500;
  console.log("\nStarting train learn rate = " +
    lrnRate.toString());
  nn.train(trainX, trainY, lrnRate, maxEpochs);
  console.log("Done ");
. . .

Notice that I made train() a method that belongs to the NeuralNet class rather than a standalone function that accepts a neural net object. Design decisions like this are often more difficult than coding implementation. Anyway, it was a good way to spend a Saturday evening.

---

I’ve always been fascinated by aircraft design. Left: The German Albatross D5 (1917) featured an early streamlined design with a spinner fairing over the propeller. The Albatross could fly at 115 mph. Center: Just 20 years later, the Vought Corsair F4U (1937) featured an inverted gull wing design to allow a huge propeller. The Corsair could fly at 445 mph. Right: And just another 20 years later, the Convair F-106 Delta Dart (1957) featured a delta shaped wing for large wing leading edge angle for speed with large surface area for lift. The F-106 could fly at 1,525 mph.

---