liveBook · Manning

Chapter 3: Embedding-based recommender systems

What you will learn in chapter 3 of AI Recommender Systems

Introduces embeddings as similarity representations for recommenders and contrasts behavioral, content-based, and LLM embedding sources.

After this chapter you will be able to

What it builds on

Section by section

Introduction

Introduces embeddings as similarity representations for recommenders and contrasts behavioral, content-based, and LLM embedding sources.

Key terms: embedding, TF-IDF, cosine similarity, collaborative filtering, user-item interactions, cold start problem, content-based embeddings, BERT, Sentence Transformers, LLM embeddings

3.1 Representations and embeddings

3.1.1 Embeddings as Compressed Representations

This section explains how embeddings compress item information into dense vectors while preserving meaningful similarity.

Key terms: embedding, dense vector, TF-IDF, sparse, similarity structure, embedding space

3.1.2 Why Embeddings Enable Similarity Search

This section explains why nearby embeddings can represent similar items and how cosine similarity and the dot product measure that closeness.

Key terms: embedding space, meaningful similarity, cosine similarity, dot product, normalized vectors, unit length

3.1.3 The many ways to create an Embedding

The section explains that different embedding methods produce different kinds of item similarity, so the choice of method should match the similarity a recommender needs.

Key terms: embedding, similarity, content-based embeddings, behavioral embeddings, collaborative filtering, user interactions, LLM embeddings, world knowledge

3.2 Collaborative filtering (CF)

This section explains how collaborative filtering uses shared user behavior to recommend items, what distinguishes it from content-based methods, and why new users and items pose a cold-start problem.

Key terms: collaborative filtering, similar tastes, content-based methods, behavior, interaction data, embeddings, shared audiences, cold start, behavioral data, initial engagement

3.3 Collaborative Filtering with Matrix Factorization

This section motivates matrix factorization as a foundation for collaborative filtering and ALS as an accessible way to study it.

Key terms: matrix factorization, collaborative filtering, Alternating Least Squares (ALS), candidate-generation method

3.3.1 How Matrix Factorization Works

This section explains how matrix factorization learns user and item vectors from feedback to predict missing ratings and reveal similarities.

Key terms: matrix factorization, user factors, item factors, latent space, dot product, missing ratings, user vector, item vector, collaborative filtering, similar factors

3.3.2 From SVD to Alternating Least Squares

This section explains why sparse ratings make SVD and imputation problematic, and how ALS learns user and item factors from observed ratings.

Key terms: Singular Value Decomposition (SVD), imputation, observed ratings, Alternating Least Squares (ALS), user factors, item factors, least squares, closed-form solution, global optimum

3.3.3 Implementing ALS

This section describes the three-step workflow for putting an ALS model into service: train and save it, load it at startup, and use it to handle requests.

Key terms: batch job, front-end system, startup script, requests

3.3.4 Training an ALS Model

The section prepares MovieLens ratings for implicit-feedback ALS, stores the interactions efficiently, and trains a model that represents users and movies as factors in a shared space.

Key terms: explicit ratings, implicit feedback, positive implicit feedback, confidence weight, user-item matrix, sparse matrix, Compressed Sparse Row (CSR), mapping dictionaries, user_map, item_map

3.3.5 Generating Recommendations

This section explains how ALS retrieval chooses between user-based recommendations and item-similarity recommendations, then combines results from multiple seed items.

Key terms: retrieve, recommend, similar_items, user vector, item vectors, seed items, max score, similarity score, merge

3.3.6 Strategies for multiple seeds

This section compares two ways to retrieve recommendations from multiple seed items and explains how merging their results changes the candidates returned.

Key terms: multi-query union pattern, seed-centroid approach, recall, max-over-seeds, average-over-seeds, precision, rank fusion

3.3.7 Using the framework

Demonstrates user-based and item-based retrieval with examples showing how behavioral data produces recommendations without genre metadata.

Key terms: user-based recommendations, behavioral data, item-based recommendations, similar embeddings, user behavior

3.3.8 Choosing Hyperparameters

Explains how to choose ALS factor count, regularization, and iterations using held-out recommendation quality rather than fixed defaults.

Key terms: factors, embeddings, validation metric, knee of that curve, regularization, overfitting, underfits, training metrics, validation performance, iterations

3.3.9 Production Considerations

This section explains where ALS fits in a recommender pipeline and how serving and retraining choices affect latency, freshness, and compute.

Key terms: ALS, retrieval model, scorer, four-stage recommender system, on demand, precompute recommendations, batch job, serving layer, latency, retraining

3.4 Embeddings from Content: BERT

The section explains why content-based embeddings complement collaborative filtering and how pretrained language models can represent an item’s meaning without relying on user interactions.

Key terms: collaborative filtering, interaction data, content-based recommendations, metadata, content-based embeddings, semantic meaning, BERT, Sentence Transformers, pre-training

3.4.1 Sentence Transformers: BERT for Similarity Search

This section explains why Sentence Transformers are used for similarity-search embeddings and how to choose a model for a particular set of items.

Key terms: BERT, Sentence Transformers, embeddings, similarity search, model name, all-MiniLM-L6-v2, similar, dissimilar

3.4.2 Creating Content Embeddings

This section describes creating semantic movie embeddings from available text and explains why their normalization simplifies similarity comparisons.

Key terms: movie descriptions, Pandas DataFrame, embedding, dense 384-dimensional vector, semantic meaning, normalized, unit length, dot product, cosine similarity

3.4.3 Finding Similar Items

This section uses content embeddings to find similar movies from one or more seed items and notes a filtering issue in the resulting recommendations.

Key terms: embeddings, content-based similarities, more like this, seed items, query vector, nearest neighbors, averaging, max aggregation, deduplication logic, filtering stage

3.5 Embeddings from Language Models

This section contrasts Sentence Transformer and LLM embeddings using movie descriptions that refer to the same film in different ways.

Key terms: Sentence Transformers, Large Language Models (LLMs), semantic similarity

3.5.1 What Makes LLM Embeddings Different?

This section contrasts the knowledge encoded in LLM embeddings with BERT-based embeddings and cautions that the extra knowledge may not improve recommendations.

Key terms: LLM embeddings, BERT-based embeddings, Sentence Transformers, entity relationships, cultural references, implicit themes

3.5.2 Using Open-Source LLM Embeddings

This section shows how to use an open-source instruction-tuned embedding model and why its results must be evaluated against a smaller model on your own data.

Key terms: Instructor, instruction prefix, open-source models, MiniLM, world knowledge, similarity scores

3.5.3 Steering Embeddings with Instructions

This section shows how changing an instruction steers recommendations from the same seed movie and explains why evaluating the resulting changes matters.

Key terms: instruction-tuned models, embedding steering, seed movie, evaluation framework, mystery genre, rivalry genre

3.5.4 When LLM Embeddings Provide Real Value

This section explains when LLM embeddings can improve recommendations and how to weigh those gains against their cost.

Key terms: LLM embeddings, world knowledge, cultural and temporal context, franchise or sequel relationships, Sentence Transformers, batch pipeline, real-time system, index size, retrieval latency

3.6 Combining Multiple Embedding Sources

This section explains how to combine retrieval from different embedding sources, adjust their weights to context, and add sources only when they improve results.

Key terms: weighted retrieval, candidates, scores, collaborative filtering, content embeddings, adaptive weighting, content, ratings, user behavior, hybrid approaches

3.7 Summary

Where this is used later

Concepts explained in this chapter

← Chapter 2: Implementing a simple recommender model Chapter 4: Evaluating what makes a good recommendation →

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.