An inner product, or dot product, combines two equal-length numerical vectors into one number. Multiply corresponding components, then add the products. The result can express a weighted feature calculation or a measure of alignment between vectors. section 8.4 section 12.2.1
Here w and x each have p components. The transpose symbol T changes the orientation of w for multiplication, not its component values. section 8.4
In a fitted support vector machine (SVM), the dot product wᵀx combines feature values with their fitted weights; adding the bias b gives the decision score. Its sign determines the predicted class. Dot products also connect the direct and support-vector prediction forms; kernels calculate transformed dot products without explicitly constructing every new feature. section 8.4 section 8.9
A weighted classification score
Let w = (6, −1), b = −5, and x = (3, 9), where the features are temperature and temperature squared on a simplified scale. The dot product is (6 × 3) + (−1 × 9) = 9. Adding the bias gives a score of 4, so the prediction is class +1. The boundary is linear in this feature space, though the rule is nonlinear in temperature alone. section 8.4
Alignment in attention
Suppose the query for “it” is [0.2, 0.8], and the keys for “trophy” and “box” are [1, 0] and [0, 1]. Their dot products with the query are 0.2 and 0.8, respectively, indicating stronger compatibility with “box.” The example assumes identity projections. Dividing by √2 gives scaled scores of approximately 0.14 and 0.57; softmax then turns them into attention weights of approximately 0.39 and 0.61. The scores before softmax are not probabilities. section 12.2.1
These simplified attention vectors make the arithmetic visible, but a single calculation does not explain the behavior of a complete transformer, which also uses multiple attention heads, feed-forward networks, and stacked layers. section 12.2


















