liveBook · Manning

Mean squared error

As explained in Math and Architectures of Deep Learning

The average of squared differences between predicted and target values.

Mean squared error (MSE) measures prediction quality by averaging squared differences between predicted and ground-truth values. Squaring removes the sign, so overestimates and underestimates with equal magnitude receive equal error; larger deviations receive disproportionately larger penalties. section 2.5.4 section 8.6

A one-value comparison

For a target of 2, predictions of 1.5 and 2.5 both produce squared error (prediction − target)^2 = 0.25. A model producing exactly its target receives error 0, while a prediction of 5.51 for target −0.8 produces squared error 39.82. The metric therefore compares parameter choices by how closely their outputs match the targets. section 2.5.4 section 2.5.4

Use MSE as the loss signal in supervised model training:

  • Compute each prediction from an input, compare it with the known ground truth, and aggregate the individual squared errors across the training data. section 1.3.5
  • Adjust weights and biases iteratively so the loss decreases; neural-network training connects this loss to gradients from backpropagation. section 8.4.6 section 8.6
  • Interpret a low training loss as close agreement on the training examples, while remembering that training error is distinct from performance on unseen data. section 1.3.5 section 8.6
Algorithm 8.5 Training a neural network

Initialize , b with random values

while 𝕃 > threshold do

  ⊳ Forward pass

        for l ← 0 to L do

            (l) = W(l) (l–1) + (l)

            (l) = σ((l))

        end for

  ⊳ Loss

        𝕃 = 1/2 ||(L) – ȳ||2

  ⊳ Gradients for the last layer

         (L) = ((L) – ȳ) ∘ (L) ∘ ( – (L))

        ▽W(L)𝕃 = (L)((L–1))T

        ▽b(L)𝕃 = (L)

  ⊳ Gradients for the remaining layers

        for l ← L – 1 to 0 do

             (l) = ((W(l+1))T (l+1)) (l) ∘ ( – (l))

            ▽W(l)𝕃 = (l) ((l–1))T

            ▽b(l)𝕃 = (l)

        end for

  ⊳ Parameter update

        W = W – r▽W𝕃

        b = b – r▽b𝕃

end while

Trace MSE through a complete neural-network loop: forward pass, loss computation, backpropagated gradients, and weight and bias updates. section 8.4.6

How Math and Architectures of Deep Learning covers it

How deep it goes
Defines squared error, works numeric examples, and connects aggregate loss to gradient-based training without detailing implementation libraries.
Examples
Uses symmetric errors around target 2, a zero-error parameter choice, and a complete neural-network training algorithm.
What it assumes
Assumes vectors, model outputs, targets, weights, biases, and basic supervised-training terminology.
What you can do after
Apply MSE to compare predictions with targets and understand its role in updating model parameters.
How it connects
Connects ground truth, supervised learning, Euclidean distance, forward propagation, backpropagation, and gradient-based optimization.
Cover of Math and Architectures of Deep Learning
liveBookmanning.com
Read Math and Architectures of Deep Learning on liveBook Get the book at manning.com

Also explained in

Compare how 13 books explain mean squared error →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Math and Architectures of Deep Learning. The section links point to the parts of the book it draws on.