When text is represented as vectors, a single score can help compare two passages or test whether a pattern appears in a passage. The inner product, also called the dot product or scalar product, produces that score: multiply corresponding components of two equal-length vectors, then add the products. Its output is one number, not another vector. section 3.3.1 section 7.2.1
Calculate one by hand and in Python
For [1, 2, 3] and [2, 3, 4], calculate 1×2 + 2×3 + 3×4 = 20. With NumPy, `v1 = np.array([1, 2, 3]); v2 = np.array([2, 3, 4]); v1.dot(v2)` gives `20`. `(v1 * v2).sum()` gives the same result and makes the multiply-then-add steps visible. section 3.3.1
Listing 3.9 Example dot product calculation
>>> v1 = np.array([1, 2, 3]) >>> v2 = np.array([2, 3, 4]) >>> v1.dot(v2) 20 >>> (v1 * v2).sum() #1 20 >>> sum([x1 * x2 for x1, x2 in zip(v1, v2)]) #2 20
Compare NumPy’s dot product with elementwise multiplication and a Python loop; all three calculate the same scalar. section 3.3.1
A dot product is the numerator of cosine similarity, not the complete similarity measure. Divide it by the product of the vectors’ magnitudes to obtain the normalized dot product. This measures directional similarity without calculating a trigonometric function directly, which is useful when text vectors have many dimensions. section 3.3 section 3.3.1
Listing 3.11 Computing cosine similarity in Python
>>> import math >>> def cosine_sim(vec1, vec2): ... dot_prod = 0 ... for x1, x2 in zip(vec1, vec2): ... dot_prod += x1 * x2 ... ... mag_1 = math.sqrt(sum([x1**2 for x1 in vec1])) ... mag_2 = math.sqrt(sum([x2**2 for x2 in vec2])) ... ... return dot_prod / (mag_1 * mag_2)
Follow the pairwise multiplication and sum, then the two magnitude calculations and final division in a plain-Python cosine-similarity function. section 3.3.1
For a search across many sentence vectors, calculate many dot products at once rather than writing a Python loop for each comparison. In one test involving 41,531 sentence vectors of 384 dimensions, the loop took about 6.3 times as long as the vectorized calculation. That is a measurement, not a guarantee: a test on memory-mapped vectors found the loop faster, so benchmark the representation you actually use. section 3.2.2
The same multiply-and-sum operation appears inside a text convolutional neural network. At each position, a kernel’s weights form a dot product with a window of numerical text values. Sliding the kernel produces a sequence of match scores; max pooling can then keep the strongest match. section 7.2.1
