liveBook · Manning

Total sum of squares

As explained in Statistics Every Programmer Needs

The sum of squares total is the sum of squared differences between observed response values and their overall mean.

The total sum of squares (SST) measures the dependent variable’s total variability without considering predictors. It adds each squared difference between an observed response and the response mean: SST = Σ(yᵢ − ȳ)². Squaring prevents positive and negative deviations from canceling. section 4.2.3

Values far from the overall mean contribute more to SST than values close to it. In the marathon-stage example, SST for stage2 is 49,636.95; the observation 184 contributes 4,549.50 because it is far below the mean of 251.45, while 242 contributes 89.30. section 4.2.3

Computing SST with NumPy

Use the response series and its mean directly: SST = np.sum((mds['stage2'] - np.mean(mds['stage2'])) ** 2). The resulting value is 49,636.950000000004. section 4.2.3

SST supplies the denominator for R²: R² = SSR / SST. In the fitted regression, SSR is 28,537.79, SSE is 21,099.16, and SST = SSR + SSE = 49,636.95. Thus, R² is approximately 0.5749. section 4.2.3 section 1.4.1

Table 4.2 The sum of squares summary for the fitted linear regression. The stage1 and stage2 columns represent, respectively, the observed values from the two mds variables. SST equals the sum of squares total, or the total variability in the dependent variable without the consideration of any predictors. The yhat column contains the predictions for the response variable from the fitted regression. SSE equals the sum of squares error, or the portion of the SST that can’t be explained by the regression; and SSR equals the sum of squares regression, or the percentage of the variance in the dependent variable explained by the model.
x
stage1
y
stage2
( y − ȳ) 2
SST
y
yhat
( y − ŷ) 2
SSE
( ŷ − ȳ) 2
SSR
152
191
3,654.20
195.78
22.88
3098.73
162
184
4,549.50
208.87
618.36
1813.33
165
193
3,416.40
212.79
391.71
1494.46
163
191
3,654.20
210.18
367.68
1,703.62
166
193
3,416.40
214.10
445.21
1,395.02
187
292
1,644.30
241.57
2,542.75
97.53
175
235
270.60
225.87
83.27
654.10
202
226
647.70
261.20
1,238.96
95.04
179
214
1,402.50
231.11
292.68
413.80
222
274
508.50
287.36
178.62
1,289.87
184
300
2,357.10
237.65
3,887.60
190.46
193
242
89.30
249.42
55.12
4.10
255
279
759.00
330.54
2,656.24
6,255.02
202
288
1,335.90
261.20
718.30
95.04
221
269
308.00
286.06
290.92
1,197.61
207
329
6,014.00
267.74
3,752.75
265.37
227
268
273.90
293.91
671.14
1,802.54
239
315
4,038.60
309.61
29.10
3,382.11
164
201
2,545.20
211.48
109.90
1,597.33
226
345
8,751.60
292.60
2,745.97
1,693.16
49,636.95
21,099.16
28,538.24

Use the sum-of-squares summary to compare observed responses with their mean, fitted predictions, residual error, and explained variation. section 4.2.3

How Statistics Every Programmer Needs covers it

How deep it goes
Derives the formula, works through individual deviations, computes SST with NumPy, and connects it to SSR, SSE, and R².
Examples
Uses marathon stage times, individual squared deviations, a NumPy calculation, and a regression sum-of-squares table.
What it assumes
Assumes a dependent variable and linear-regression context while developing the mechanics of variance decomposition from the formula upward.
What you can do after
Diagnose model fit with NumPy by calculating total variability and decomposing it into explained and unexplained portions.
How it connects
Connects total variability with linear regression, regression sum of squares, error sum of squares, R², and adjusted R².

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Statistics Every Programmer Needs. The section links point to the parts of the book it draws on.