Absolute trick

As explained in Grokking Machine Learning

A book-specific line-adjustment technique that updates a model's parameters with sign-based corrections derived from whether a point lies above or below the line.

The absolute trick adjusts a regression line using only whether an observed point lies above or below the line. It changes the slope and y-intercept by small, sign-based amounts rather than using the size of the prediction error. section 3.3

Apply it to one data point at a time:

• Calculate the predicted value from the current slope and y-intercept. section 3.3
• If the observed value is higher than the prediction, increase the slope by learning rate × input and increase the intercept by learning rate. section 3.3
• Otherwise, decrease the slope by learning rate × input and decrease the intercept by learning rate. section 3.3

Moving a house-price line

For a house-price model, a point above the line asks the line to rotate and move upward; a point below it asks the line to rotate and move downward. Repeating these small corrections gradually produces a better-fitting line. section 3.3

Figure 3.6 An illustration of the linear regression algorithm. We start at the top left with a random line and end in the bottom left with a line that fits the dataset well. At each stage, two things happen: (1) we pick a random point, and (2) the point asks the line to move closer to it. After many iterations, the line will be in a good position. This figure has only three iterations for illustrative purposes, but in real life, many more iterations are needed.

Visualizes the repeated process of selecting a point and moving a random line closer to the dataset. section 3.3

How Grokking Machine Learning covers it

How deep it goes

Defines the update procedure and code, but defers its broader justification to gradient descent.

Examples

Uses house-price points, above-or-below line cases, and visual line-fitting iterations.

What it assumes

Assumes a basic line equation and teaches slope, intercept, and line movement before updating parameters.

What you can do after

Apply it via a short function that updates a regression line from individual labeled points.

How it connects

Connects linear regression with slope, y-intercept, prediction error, learning rate, and gradient descent.

liveBookmanning.com

Read Grokking Machine Learning on liveBook Get the book at manning.com

Related concepts

• linear regressionused by · Linear regression is a statistical modeling method that estimates a continuous response as a linear combination of predictor variables and fitted coefficients.
• lineuses · A line is a one-dimensional straight geometric set extending indefinitely in both directions, commonly represented by a linear equation.
• learning rateconfigured by · A learning rate is a step-size parameter that controls the magnitude of updates made by an iterative learning or optimization process.

This explainer was generated by Manning's knowledge graph from the text of Grokking Machine Learning. The section links point to the parts of the book it draws on.