Restricted Boltzmann Machines are Difficult to Explain

A restricted Boltzmann machine (RBM) is a software system that can be used in some machine learning algorithms. RBMs are very difficult to explain concisely. In the diagram below, there is an RBM with six so-called visible nodes. Each visible node has a value of 0 or 1. There are three hidden (or “latent”) nodes. Each hidden node also has a 0 or 1 value.


The “p” values shown in the hidden nodes indicate that the node values in an RBM are stochastic meaning they are 1 with some probability. For example, the first hidden node has value 1 with p = 0.354 and in this case has a 0 value. Notice the third hidden node has p = 0.602 so you’d expect the node value to be 1, but because node values are stochastic, there’s a chance the node value can be 0 as shown.

Each of the bidirectional red arrows in the diagram represents a constant called a weight. The unidirectional green arrows represent constants called biases. The probability value in any of the three hidden nodes is calculated using the values in the visible nodes, the weights, and the bias associated with the hidden node.

The reason why RBMs are so difficult to explain is that there are many variations of RBMs, and RBMs are used in several different ways.

I coded up a demo of one particular use of RBMs where the goal is to find a meta relation between the visible nodes and the hidden nodes. I created 20 data items. Each data item corresponds to a person’s rating of six movies and so has six 0-or-1 values. The six 0/1 values correspond to: The Fifth Element, Forbidden Planet, The Hangover, Meet the Parents, Ben Hur, and Gladiator. A 1 value means the person likes the movie and a 0 means the person didn’t like the movie.

I created data so that people like SciFi (the first two movies), and/or Comedies (the next two movies), and/or History (the last two movies).

The demo creates and trains an RBM using the demo data.

Anyway, this blog post barely scratches the surface of what RBMs are. Restricted Boltzmann machines: very, very difficult to explain. The Wikipedia entry on the topic is pretty decent but certainly isn’t light reading.


This entry was posted in Machine Learning. Bookmark the permalink.