Correlation quantifies how paired values in two sequences vary together. Pearson correlation centers each sequence by subtracting its mean, then compares their shared variation after scaling by both sequences’ magnitudes. section 7.2.3 section 4.3.1
The coefficient requires sequences of equal length. A value of 1.0 indicates perfect self-correlation, while values nearer zero indicate weaker linear alignment; interpretation depends on the data being compared. section 7.2.3
Computing Pearson correlation
A short NumPy function centers two arrays and returns their normalized dot product. For a repeating array a and steadily increasing array b, corr(a, b) is 0.316; corr(a, a) is 1.0. section 7.2.3
Correlation takes several practical roles:
- Relate features during dimensionality reduction: covariance and correlation reveal redundancy between dimensions, helping PCA find axes that preserve variance while reducing dependence. section 4.3.1
- Inspect neural-network representations: correlating hidden weights with character position exposes units whose values track where a name occurs in the sequence. section 8.2.3
- Measure local pattern matches in CNNs: a kernel’s weights and a text window produce a dot product or correlation score, which can be evaluated across sliding windows. section 7.2.1
Listing 7.1 A function to compute Pierson correlation
>>> import numpy as np >>> def corr(a, b): ... """ Compute the Pearson correlation coefficient R """ ... a = a - np.mean(a) ... b = b - np.mean(b) ... return sum(a * b) / np.sqrt(sum(a*a) * sum(b*b)) >>> a = np.array([0, 1, 2, 0, 1, 2, 0, 1, 2]) >>> b = np.array([0, 1, 2, 3, 4, 5, 6, 7, 8]) >>> corr(a, b) 0.316... >>> corr(a, a) 1.0
Inspect the complete NumPy implementation and its output on repeating and increasing sequences. section 7.2.3
