chapter five

5 A continuous approach to splitting points: Logistic regression

This chapter covers

The difference between hard assignments and soft assignments.
Activation functions such as the step function vs the sigmoid function.
Discrete perceptrons vs continuous perceptrons.
The logistic regression algorithm for classifying data.
Coding the logistic regression algorithm in Python.
Using the softmax function to build classifiers for more than two classes.

In the previous chapter, we built a classifier that determined if a sentence was happy or sad. But as you can imagine, there are sentences that are happier than others. For example, the sentence “I’m good.” and the sentence “Today was the most wonderful day in my life!” are both happy, yet the second one is much happier than the first one. Wouldn’t it be nice to have a classifier that not only predicts if sentences are happy or sad, but that actually gives us a rating for how happy sentences are? Say, a classifier that tells us that the first sentence is 60% happy and the second one is 95% happy? In this section we will define the logistic regression classifier, which does precisely that. This classifier assigns a score from 0 to 1 to each sentence, in a way that the happier a sentence is, the higher the score it is assigned.

5.1 Logistic Regression (or continuous perceptrons)

5.1.1 A probability approach to classification - The sigmoid function

5.1.2 The error functions - Absolute, square, and log loss

5.1.3 More on the log loss error function

5.2 Reducing the log loss error: The logistic regression trick

5.2.1 An example with a discrete perceptron and a continuous perceptron

5.2.2 A second example with a discrete perceptron and a continuous perceptron

5.2.3 Moving the line to fit the points - The logistic regression algorithm

5.2.4 Coding the logistic regression algorithm

5.2.5 The logistic regression algorithm in Turi Create

5.3 Classifying into multiple classes - The softmax function

5.4 Summary