Restricted Boltzmann Machines using C#

I wrote an article titled ” Restricted Boltzmann Machines using C#” in the June 2017 issue of MSDN Magazine. https://msdn.microsoft.com/en-us/magazine/mt809120.

Restricted Boltzmann machines (RBMs) are fascinating software systems. Technically they’re quite simple, but conceptually they’re quite deep. In my article I present an example of an RBM that analyzes movie preference data for a set of 12 people. The raw data is:

(1, 1, 0, 0, 0, 0) // A
(0, 0, 1, 1, 0, 0) // B
(0, 0, 0, 0, 1, 1) // C
(1, 1, 0, 0, 0, 1) // noisy A
(0, 0, 1, 1, 0, 0) // B
(0, 0, 0, 0, 1, 1) // C
(1, 0, 0, 0, 0, 0) // weak A
(0, 0, 1, 0, 0, 0) // weak B
(0, 0, 0, 0, 1, 0) // weak C
(1, 1, 0, 1, 0, 0) // noisy A
(1, 0, 1, 1, 0, 0) // noisy B
(0, 0, 1, 0, 1, 1) // noisy C

Each row represents one person. The six 0-or-1 values in each row indicate if the person likes (1) or dislikes (0) six movies. The six movies, in order, are “Alien”, “Inception”, “Spy”, “EuroTrip”, “Gladiator”, Spartacus”.

If you examine the data, you’ll see that there are three basic types of people: those who like science fiction films (Type A), those who like comedies (Type B), and those who like history films (Type C). There are variations of these types of people.

Anyway, you can feed this data to an RBM (without giving the RBM any hints as I’ve done above) and the RBM will figure out that there are in fact three kinds of people. These three types of people, A, B, C, get encoded as (1,1,0) and (1,0,1) and (0,1,1) respectively. Quite interesting!

And if you feed an encoding such as (1,0,1) to the RBM, it will predict which movies will be liked: 0,0,1,1,0,0.

The technical details of RBMs are very interesting too, in particular the training algorithm that gives life to an RBM. RBMs can be used in several ways, either by themselves, or as components in a larger, more complex machine learning system.

Advertisements
This entry was posted in Machine Learning. Bookmark the permalink.

2 Responses to Restricted Boltzmann Machines using C#

  1. Thanks to your articles over the years i’ve been able to understand many fascinating topics which were previously impervious for me behind the complex math, and i’m sad to say – obscurantism in scientific papers. Thank you for that. I’m curious about a couple of things:
    Can RBMs be used for continuous numeric features or just binary features (i’m interested in PCA alternatives)? Is it possible to train them using PSO, Firefly and similar algorithms? As an aside, i saw your talk “Data Clustering for Developers”, and i’m wondering is your code for the C-means fuzzy clustering algorithm available online?

    • It is in fact possible to have a RBM that uses real-valued data. Here’s a good video by G Hinton:

      And yes, in principle it is possible to train an RBM using swarm optimization.

      Sorry, I don’t have the code associated with my data clustering video.

      James M

Comments are closed.