A projection maps a vector onto a chosen line or axis, preserving only its component along that direction. In practical machine learning, this turns a high-dimensional vector into a score that can support comparison or classification. section 4.2.1
For two-class linear discriminant analysis, the useful direction runs between the toxic and nontoxic class centroids. Projecting each TF–IDF vector onto that centroid axis produces a toxicity score; normalizing the scores and applying a cutoff converts them into predictions. section 4.2.1
The same operation appears in attention systems:
- A linear projection is a dot product or matrix multiplication that transforms embedding vectors into query, key, and value vectors. section 9.1.2
- Dot products between queries and keys create attention scores; scaling by the key dimension improves numerical stability before softmax produces weights. section 9.1.2
Projecting comments onto a toxicity axis
Compute toxic and nontoxic TF–IDF centroids, subtract them to form an axis, and calculate every comment's score with one vectorized dot product. A MinMaxScaler and a 0.5 cutoff then produce manual predictions. The resulting classifier illustrates both the usefulness and limitation of the approach: training accuracy reaches 0.999, while held-out accuracy falls to 0.554. section 4.2.1 section 4.2.1
>>> mask = comments.toxic.astype(bool).values #1 >>> toxic_centroid = tfidf_docs[mask].mean(axis=0) #2 >>> nontoxic_centroid = tfidf_docs[~mask].mean(axis=0) #3 >>> centroid_axis = toxic_centroid - nontoxic_centroid >>> toxicity_score = tfidf_docs.dot(centroid_axis) #4 >>> toxicity_score.round(3) array([-0.008, -0.022, -0.014, ..., -0.025, -0.001, -0.022])
Inspect the NumPy calculation that builds the centroid axis and projects all TF–IDF vectors into toxicity scores. section 4.2.1
