I Give a Talk on Logistic Regression

I consider logistic regression to be the “Hello World” of machine learning. I gave a talk to a group of (mostly) engineers, where I explained what logistic regression (LR) is, explained how LR works, and described different ways to perform an LR analysis.


1. Logistic regression is a technique for binary prediction. For example: predict if a person is a political conservative (0) or liberal (1) based on their age, income, years of education.

2. Logistic regression is basically a prediction equation with special constants called weights and biases.

3. The prediction equation is p = 1.0 / (1.0 + e^-z) where z = b0 + (b1)(x1) + (b2)(x2) + (b3)(x3) = . . .

The b0 is the bias, the b1 through bn are the weights, and the x1 trough xn are the input predictor values. The p value will always be between 0 and 1. If p is less than 0.5 the prediction is class “0” otherwise the prediction is class “1”.

4. Determining the values of the weights and biases for an LR model is called training. There are many training algorithms including gradient ascent to maximize log-likelihood (the most common), gradient descent to minimize squared error (most similar to neural network training), iterated Newton-Raphson, L-BFGS, and swarm optimization.

5. You can do an LR analysis by writing raw code (my preferred technique because you get total control), use the R language glm() function (very easy), use the Azure Machine Learning service, use the internal-Microsoft TLC library (the engine underneath Azure ML), or use the CNTK code library (most useful for enormous datasets).

During my talk, I was reminded how difficult it can be to explain something simple. There’s a delicate balance between being too brief or presenting too much background information and extra details.

But there’s no better way for me to be sure I completely understand a machine learning topic than to give a talk on the topic.

This entry was posted in Machine Learning. Bookmark the permalink.

4 Responses to I Give a Talk on Logistic Regression

  1. Sidrah Sid says:

    Hi! Sir….i tried to ask you for a favor about one of your post….i want to understand that code completely in the post “Getting Word Synonyms using WordNet and C#” can you plz provide me the complete code along with wordnet class to as i have to use it in one my model…will be thankful to u for that….https://jamesmccaffrey.wordpress.com/2013/06/03/getting-word-synonyms-using-wordnet-and-c/

  2. Sidrah Sid says:

    But Sir you said that you have written that code for VS magazine so if you published that in the magazine can you plz then provide me the link for that …i really need it to get things

    • No; never published that code in VS Magazine or anywhere else. You should be able to find other examples all over the Internet, or just code it yourself — it’s not terribly difficult.

Comments are closed.