liveBook · Manning

Inner product

Also called dot product

An inner product is a scalar-valued operation on two vectors that generalizes the dot product, which for real coordinate vectors is the sum of products of corresponding components.

How 19 Manning books explain it

AI-Powered Search

A dot product is the sum of pairwise products of corresponding components of two vectors.

How it is used
Uses dot products to score term matches and compare dense embeddings in search.
How deep it goes
Works through pairwise multiplication, magnitude, normalization, and practical scoring implementations.
Examples
Scores apple-juice term vectors and compares sentence embeddings for weather and vehicle phrases.
What you can do after
Apply it via Sentence Transformers to compare normalized phrase embeddings and rank their similarities.

Read how AI-Powered Search explains inner product →

Geometry for Programmers

The dot product is a scalar product of two vectors formed by summing the products of corresponding coordinates.

How it is used
Uses directional alignment for projections, surface shading, smooth arc-line joins, and point-to-segment projection.
How deep it goes
Defines coordinate and N-dimensional forms, derives projection and angle meaning, and implements SymPy and 2D uses without deriving broader vector-product theory.
Examples
Demonstrates coordinate multiplication, SymPy vectors, sphere shading, arc-line smoothness, and projection onto 2D segments.
What you can do after
Apply it via SymPy or a 2D helper to compute alignment, lighting coefficients, orthogonality constraints, and segment projections.

Read how Geometry for Programmers explains inner product →

Math and Architectures of Deep Learning

A scalar computed from two vectors by multiplying corresponding components and summing them, used here to measure one vector's projection along another.

How it is used
Expresses weighted sums compactly for linear models, document similarity, and neural-network layers.
How deep it goes
Defines componentwise computation, develops projection and orthogonality intuition, then connects it to matrix operations and PyTorch.
Examples
Uses cat-brain weighted sums, document word vectors, physical push and displacement, and PyTorch vector tensors.
What you can do after
Apply it via PyTorch to compute vector products and interpret zero, positive, and negative results geometrically.

Read how Math and Architectures of Deep Learning explains inner product →

Timeless Algorithms, The Foundational Ideas

A dot product is an operation that maps two equal-length numerical vectors to the sum of their pairwise coordinate products.

How it is used
Uses dot products to calculate SVM decision scores and attention compatibility.
How deep it goes
Works through component-wise arithmetic in both settings, while simplifying attention vectors rather than reproducing a complete transformer.
Examples
Calculates a temperature-based classification and compares pronoun-query compatibility with trophy and box keys.
What you can do after
Evaluate and understand SVM decision scores and attention compatibility through dot-product calculations.

Read how Timeless Algorithms, The Foundational Ideas explains inner product →

Classic Computer Science Problems in Swift: Essential techniques for practicing programmers

A dot product is the scalar obtained by multiplying corresponding components of two equal-length vectors and summing the products.

How it is used
Combines neuron inputs with weights during feed-forward computation and next-layer weights with deltas during backpropagation.
How deep it goes
Shows a Swift implementation and two network call sites, but does not develop the underlying vector mathematics or derive training formulas.
Examples
Uses an Accelerate-backed `dotProduct` helper, a neuron's `output(inputs:)`, and a hidden layer's delta calculation.
What you can do after
Apply it via Accelerate's `vDSP_dotprD` to implement neuron outputs and hidden-layer delta calculations.

Read how Classic Computer Science Problems in Swift: Essential techniques for practicing programmers explains inner product →

Natural Language Processing in Action, Second Edition

A dot product is the scalar obtained by multiplying corresponding components of two equal-length vectors and summing the products.

How it is used
Uses dot products to score text-vector similarity and kernel matches in NLP.
How deep it goes
Explains pairwise multiplication and summation, then implements normalized similarity and benchmarks vectorized search.
Examples
Compares calculations for two three-element vectors and searches 41,531 sentence vectors.
What you can do after
Apply it via NumPy to calculate vector products and cosine similarity.

Read how Natural Language Processing in Action, Second Edition explains inner product →

Statistics Every Programmer Needs

A dot product is an algebraic operation that multiplies corresponding elements of two equal-length sequences and sums the products.

How it is used
Uses vector–matrix multiplication to forecast future state distributions in Markov analysis.
How deep it goes
Defines the operation, explains its Markov interpretation, and demonstrates repeated and direct two-period calculations.
Examples
Demonstrates cellular-market share forecasts across periods using NumPy vectors and transition matrices.
What you can do after
Apply it via NumPy to calculate and iterate Markov state-distribution forecasts.

Read how Statistics Every Programmer Needs explains inner product →

Rearchitecting LLMs

A dot product is a scalar operation between two equal-length vectors that measures their directional alignment.

How it is used
Scores Query–Key alignment to determine Value contributions in attention.
How deep it goes
Explains alignment scores and their softmax normalization into weights, without deriving the operation mathematically.
Examples
Compares the Query for “spring” with the Key for “mountain” to illustrate a high alignment score.
What you can do after
Understand how alignment scores become attention weights for mixing Value vectors.

Read how Rearchitecting LLMs explains inner product →

Build a Robo-Advisor with Python (From Scratch)

The dot product is an operation that sums the pairwise products of corresponding components of two vectors to produce a scalar.

How it is used
Computes weighted portfolio returns and expected values for allocation choices in goals-based dynamic programming.
How deep it goes
Defines the vector operation and implements it with NumPy, stopping short of broader matrix-multiplication details.
Examples
Uses FE, WMT, and AAPL weights for portfolio return, then evaluates wealth-state transitions against future values.
What you can do after
Apply it via NumPy to calculate portfolio returns and expected values for discrete allocation decisions.

Read how Build a Robo-Advisor with Python (From Scratch) explains inner product →

Grokking AI Algorithms, Second Edition

A dot product is an operation on two equal-length vectors that multiplies corresponding components and sums the resulting products.

How it is used
Replaces repeated weighted-sum loops and scores alignment in neurons, self-attention, and convolutional filters.
How deep it goes
Defines pairwise multiplication and summation, then works through attention and convolution examples without formal proofs.
Examples
Uses neuron inputs and weights, Query-Key token scores, and a 3×3-style filter scanning image pixels.
What you can do after
Apply it to calculate weighted sums, attention relevance scores, and convolution responses by hand.

Read how Grokking AI Algorithms, Second Edition explains inner product →

Deep Learning with Python, Third Edition

An inner-product operation used as a similarity score between vectors.

How it is used
Uses vector products as similarity scores in attention and as the basic operation underlying matrix multiplication.
How deep it goes
Explains vector, matrix-vector, matrix, and higher-dimensional products with shape rules, but avoids deeper mathematical derivation.
Examples
Demonstrates NumPy matmul, Python's @ operator, naive products, einsum attention, and parameterized query-key-value projections.
What you can do after
Apply it via NumPy, Keras operations, Python's @ operator, or einsum-based attention implementations.

Read how Deep Learning with Python, Third Edition explains inner product →

AI Recommender Systems

A dot product is an operation that multiplies corresponding components of two vectors and sums the resulting products.

How it is used
Measures embedding similarity, latent user-item preference, and behavioral similarity for recommender-system ranking.
How deep it goes
Defines the operation, compares it with cosine similarity, and applies it in NumPy and matrix-factorization examples.
Examples
Uses normalized embeddings, movie ratings, user-item matrices, and comic-consumption patterns as concrete applications.
What you can do after
Apply it via NumPy to compute similarity, compare factor vectors, and rank recommendations.

Read how AI Recommender Systems explains inner product →

Advanced Algorithms and Data Structures

The dot product is a scalar operation on vectors equal to the sum of the products of corresponding coordinates and related to their angle.

How it is used
Uses dot products to eliminate parameters while testing whether two planar line segments intersect.
How deep it goes
Develops perpendicular-vector construction, algebraic elimination, implementation, and endpoint handling without general inner-product theory.
Examples
Uses vectors between segment endpoints, a quarter-turned vector, scaling factors, and shared-endpoint cases.
What you can do after
Apply vector arithmetic to compute segment parameters and diagnose whether two segments intersect.

Read how Advanced Algorithms and Data Structures explains inner product →

Knowledge Graphs and LLMs in Action

A dot product is the scalar obtained by multiplying corresponding components of two equal-length vectors and summing the results.

How it is used
Scores candidate user–movie links after heterogeneous graph encoding.
How deep it goes
Defines the scoring role and shows its classifier call, but does not derive the arithmetic or explain optimization internals.
Examples
Uses user–movie ratings, genre features, and a PyTorch DotProduct classifier in an encoder–decoder implementation.
What you can do after
Apply it via a DotProduct classifier to score candidate links between learned user and movie embeddings.

Read how Knowledge Graphs and LLMs in Action explains inner product →

Deep Learning with JAX

A linear algebra operation that multiplies corresponding elements of two arrays and sums the products.

How it is used
Uses dot products for single-vector calculations and as the shape-sensitive core of batched neural-network predictions.
How deep it goes
Defines the operation, demonstrates jnp.vdot, and diagnoses a batch-shape failure without deriving contraction internals.
Examples
Uses two three-element vectors, random arrays of vectors, and a batch of 32 flattened 28×28 images.
What you can do after
Apply it via JAX's jnp.vdot, recognize incompatible batch shapes, and prepare single-item functions for vmap.

Read how Deep Learning with JAX explains inner product →

Machines that Think

A dot product is a multiplication and summation operation on two vectors that produces a scalar measuring their alignment.

How it is used
Compares each token’s query with sequence keys to assign relevance scores in self-attention.
How deep it goes
Defines the operation as vector multiplication yielding a scalar and explains how alignment affects relevance scores.
Examples
Compares the query for “rises” with the keys for “sun” and “the.”
What you can do after
Understand how query-key alignment contributes to token relevance scores before softmax converts them into attention weights.

Read how Machines that Think explains inner product →

CUDA for LLMs

An inner product combines two vectors into a scalar, including by summing corresponding elementwise products in the standard case.

How it is used
Computes output values in matrix multiplication, convolution, and matrix-vector multiplication by accumulating products of paired elements.
How deep it goes
Traces scalar accumulation through naive CPU and CUDA kernels, without developing optimized inner-product algorithms.
Examples
Shows matrix rows paired with columns, sliding filter windows, and matrix rows paired with an input vector.
What you can do after
Build a naive CUDA output kernel by assigning each output to a thread that accumulates its dot product.

Read how CUDA for LLMs explains inner product →

Understanding AI

A dot product is a scalar obtained by multiplying corresponding components of two vectors and summing the products.

How it is used
Calculates relevance scores between query and key vectors within attention.
How deep it goes
Defines componentwise multiplication and summation, then applies the result to attention without deriving vector arithmetic.
Examples
Uses token matching in a sentence to show how “it” relates most strongly to “performance.”
What you can do after
Understand how attention converts query-key similarity into relevance weights.

Read how Understanding AI explains inner product →

Retrieval Augmented Generation, The Foundational Ideas

An inner product is an operation that maps two vectors to a scalar expressing their algebraic alignment.

How it is used
Uses inner products to compare query and database vectors, then motivates approximate nearest-neighbor search at enterprise scale.
How deep it goes
Defines brute-force cost and cosine interpretation, but stops short of deriving ANN algorithms or implementation details.
Examples
Uses Netflix’s 6,500-movie catalog, Amazon’s 200-million-item catalog, and normalized query-document vectors.
What you can do after
Evaluate/understand when brute-force vector comparison becomes impractical and why cosine similarity or ANN is used.

Read how Retrieval Augmented Generation, The Foundational Ideas explains inner product →

Also covered in

What to know first

Related concepts

Questions about inner product

What is inner product?

An inner product is a scalar-valued operation on two vectors that generalizes the dot product, which for real coordinate vectors is the sum of products of corresponding components.

Which Manning books explain inner product?

AI-Powered Search, Geometry for Programmers, Math and Architectures of Deep Learning, Timeless Algorithms, The Foundational Ideas, Classic Computer Science Problems in Swift: Essential techniques for practicing programmers, Natural Language Processing in Action, Second Edition, Statistics Every Programmer Needs, Rearchitecting LLMs and 11 more.

What should I know before learning inner product?

Manning books that teach inner product usually introduce embedding, transformer architecture, token and vector space first.

What is inner product a kind of?

Inner product is a kind of scalar. A scalar is a single numerical quantity with no vector or multidimensional components.

What is inner product part of?

Inner product is part of matrix-matrix multiplication. Matrix multiplication is an operation on compatible matrices in which each entry of the product is the dot product of a row of the first matrix and a column of the second.

What is inner product also called?

Inner product is also called dot product.

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.