The inner product, usually called the dot product, turns two vectors of equal length into one scalar. For vectors a = [a₁,…,aₙ] and b = [b₁,…,bₙ], a · b = Σᵢ aᵢbᵢ: multiply corresponding components, then add the products. Equal dimensions are required. This compact notation replaces a long weighted sum such as x₀w₀ + x₁w₁ + b with x · w + b, regardless of how many features a model has. section 2.5.2
Geometrically, a · b = ‖a‖‖b‖cos(θ), where θ is the angle between the vectors. It captures how much one vector lies along the other: aligned vectors produce a large positive value, perpendicular vectors produce zero, and oppositely directed vectors produce a negative value. Vectors with dot product zero are orthogonal, meaning that one has no component in the other’s direction. section 2.5.6 section 2.6
Feature similarity
Represent each document by a vector of word-feature values. Two documents with no shared relevant words can have dot product 0, while documents sharing many relevant words have a larger dot product. The dot product therefore provides a quantitative first estimate of document similarity; a larger value loosely indicates greater agreement between feature vectors. section 2.5.6 section 4.1
A dot product is the unit computation behind larger linear-algebra operations. Matrix–vector multiplication computes one dot product per matrix row, producing weighted-sum outputs for many input instances. In a neural-network linear layer, each output neuron similarly forms a dot product between its incoming activations and its weight vector, then incorporates bias and activation processing. section 2.5.3 section 8.3.1
Figure 2.7 Vector components and dot product
|
(a) Components of a 2D vector along coordinate axes. Note that ||
(b) Dot product as a component of one vector along another |
This diagram links component projections in two dimensions to the formula a · b = ‖a‖‖b‖cos(θ). section 2.5.6
PyTorch computation
For a = [1, 2, 3] and b = [4, 5, 6], use torch.dot(a, b) to compute 1×4 + 2×5 + 3×6. For perpendicular vectors [1, 0] and [0, 1], torch.dot returns zero. section 2.7.2
