chapter five

5 Using lines to split our points: The perceptron algorithm

 

This chapter covers

  • What is classification?
  • Sentiment analysis: How to tell if a sentence is happy or sad using machine learning.
  • How to draw a line that separates points of two different colors.
  • What is a perceptron?
  • What is the perceptron algorithm?
  • How to measure the error of a perceptron model.
  • Coding the perceptron algorithm in Python and Turi Create.

In this chapter we learn a new branch of machine learning called classification. Classification models are very much like regression models, in that their aim is to predict the labels of a dataset based on the features. The difference is that regression models aim to predict a number, while classification models aim to predict a state, or a category. Classification models are often called classifiers, and we’ll use the terms interchangeably. Many classifiers predict one of two possible states (often yes/no), although it is possible to build classifiers that predict among a higher number of possible states. The following are popular examples of classifiers.

  • A recommendation model that predicts if a user will watch a certain movie or not.
  • An email model that predicts if an email is spam or ham.
  • A medical model that predicts if a patient is sick or healthy.
  • An image recognition model that predicts if an image contains an automobile, a bird, a cat, or a dog.
  • A voice recognition model that predicts if the user said a particular command or not.

5.1    The problem: We are on an alien planet, and we don’t know their language!

5.1.1    A slightly more complicated planet

5.1.2    Does our classifier need to be correct all the time? No

5.1.3    A more general classifier and a slightly different way to define lines

5.1.4    The step function and activation functions - a condensed way to get predictions

5.1.5    What happens if I have more than 2 words? General definition of the perceptron classifier

5.1.6    The bias, the y-intercept, and the inherent mood of a quiet alien

5.2    How do we determine if a classifier is good or bad? The error function

5.2.1    How to compare classifiers? The error function

5.3    How to find a good classifier? The perceptron algorithm

5.3.1    The perceptron trick: A way to slightly improve the perceptron

5.3.2    Repeating the perceptron trick many times: The perceptron algorithm

5.3.3    Gradient descent

5.3.4    Stochastic and batch gradient descent - the general version of the perceptron algorithm

5.4    Coding the perceptron algorithm

5.4.1    Coding the perceptron trick

5.4.2    Coding the perceptron algorithm

5.4.3    Coding the perceptron algorithm using Turi Create

5.5    Applications of the perceptron algorithm

5.7    Summary