Q-Learning Using C#

I wrote an article titled “Introduction to Q-Learning Using C#” in the August 2018 issue of Microsoft MSDN Magazine. See https://msdn.microsoft.com/en-us/magazine/mt829710.

Q-learning is a technique from the field of Reinforcement Learning (RL). When I think of the field of machine learning, I usually categorize techniques into: supervised learning (you create model where you have training data with known, correct labels), unsupervised learning (your data does not have labels), and reinforcement learning (you create a model using a reward function of some sort rather than by using data).

In the article, I show how to use Q-learning to solve a simple maze. Q-learning uses a clever algorithm that is based on a mathematical idea called the Bellman equation. Although the ideas are quite deep, implementing a Q-learning algorithm for a maze is fairly simple.

Of course, even though solving a maze programmatically is interesting, it isn’t terribly useful. Q-learning can be used for problems where there are a set of discrete states and you have knowledge of how/if you can transition from one state to another. In the case of a maze, each state is a location in the maze. The transitions are defined as allowable moves (moving from one position to another in the maze).

Because Q-learning is iterative, involving trial and error, Q-learning is not applicable for problems such as teaching a self-driving automobile how to navigate around a city’s streets (unless you have a sophisticated simulation available).



On a recent trip to Las Vegas for a conference, I noticed many driverless cars (Aptiva, operated by Lyft) running up and down the Strip. Notice the black BMW with red wheels — there’s nobody inside!

Advertisements
This entry was posted in Machine Learning. Bookmark the permalink.

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s