Suppose a data point has several numeric attributes, but one varies by hundreds while another varies by fractions. In k-means, the attribute with larger differences can dominate which centroid appears closest. Normalization puts attributes on comparable scales before clustering. It also prepares attributes recorded on different scales for neural network classification. section 6.2 section 7.6
Two column-by-column approaches are:
- Z-score normalization: subtract the column mean from each value, then divide by the column’s standard deviation. K-means normalizes its points this way before creating random centroids. section 6.2
- Feature scaling: map each value to 0–1 with (value − column minimum) / (column maximum − column minimum). This scale matches the range of the classifier’s sigmoid neuron outputs. section 7.6
private func zscoreNormalize() {
for dimension in 0..<Int(PointType.numDimensions) {
for (index, zscore)
in dimensionSlice(dimension).zscored.enumerated() {
points[index].dimensions[dimension] = zscore
}
}
}
The Swift k-means normalizer takes one dimension at a time, computes its z-scores, and writes them back into the points. section 6.2
/// assumes all rows are of equal length
/// and feature scale each column to be in the range 0–1
func normalizeByFeatureScaling(dataset: inout [[Double]]) {
for colNum in 0..<dataset[0].count {
let column = dataset.map { $0[colNum] }
let maximum = column.max()!
let minimum = column.min()!
for rowNum in 0..<dataset.count {
dataset[rowNum][colNum] = (dataset[rowNum][colNum]
- minimum) / (maximum - minimum)
}
}
}
The feature-scaling function finds each column’s minimum and maximum, then updates the original [[Double]] dataset through an inout parameter. section 7.6
Preparing classification data
The Iris CSV contains four numeric measurements per flower; the wine CSV contains thirteen numeric attributes per sample. Both are examples of multi-attribute data to prepare before neural network classification. Normalize each numeric column against values from that same column, not against values from unrelated attributes. section 7.6 section 6.2
The feature-scaling implementation assumes a nonempty dataset with equal-length rows. Its formula also divides by maximum minus minimum, so a column whose values are all identical needs separate handling. section 7.6
