Ordinary least squares chooses the coefficients of a polynomial by minimizing the total squared discrepancy between its values and the supplied data points. Squaring distributes error across the points and makes an isolated outlier less likely to dominate. section 6.2.2
For a polynomial of degree M, the method builds a matrix from sums of powers of the input values and a right-hand vector from sums of outputs multiplied by those powers. Solving the resulting linear system produces the coefficients. section 6.2.3
The same construction supports several useful cases:
- Degree 0 computes a mean value. section 6.2.2
- Degree 1 produces a linear trend, while degrees 2 and 3 produce quadratic and cubic approximations. section 6.2.2
- With one point repeated in the data, its squared error receives extra weight, bringing the fitted curve closer without guaranteeing exact interpolation. section 6.4.2
A rowing-time trend
Five 2000-meter rowing times are fitted with a degree-one polynomial. Python and SymPy construct the least-squares matrix, assemble the right-hand vector, solve for a0 and a1, and turn the result into a linear trend. The resulting plot makes progress—or an apparent reversal—easy to inspect. section 6.2.3
Listing 6.1 Computing the approximating polynomial
LS = Matrix([[sum([xs[i]**(row + column) #1
for i in range(N)]) #1
for column in range(M+1)] #1
for row in range(M+1)]) #1
B = Matrix([sum([ys[i]*(xs[i]**row) #2
for i in range(N)]) #2
for row in range(M+1)]) #2
a0, a1 = symbols('a0 a1')
Pol = Matrix([a0, a1]) #3
solution = solve(LS*Pol - B, (a0, a1)) #4
a = [value for key, value in solution.items()] #5
Inspect the complete coefficient-computation listing, including matrix construction, symbolic variables, solving, and extraction of coefficient values. section 6.2.3
