chapter three

3 Embedding-based recommender systems

 

This chapter covers

  • What embeddings are and why they're fundamental to modern recommenders
  • Creating embeddings from user behavior with collaborative filtering
  • Using pre-trained models (BERT) for content-based embeddings
  • Leveraging LLM embeddings for semantic understanding
  • Combining multiple embedding sources for hybrid recommendations

In Chapter 2, we built a simple content-based recommender using TF-IDF and cosine similarity. TF-IDF works, but it has two critical limitations: it treats "adventure" and "adventurous" as completely different words (no semantic understanding), and it ignores user behavior entirely—a user who always reads sports articles gets the same recommendations as someone who never does. Embeddings solve both problems. An embedding is a dense, low-dimensional vector representation that captures similarity. Items with similar embeddings are similar in some meaningful way—and crucially, that "way" depends on how the embedding was created.

This chapter explores three fundamentally different sources of embeddings:

3.1 Representations and embeddings

3.1.1 Embeddings as Compressed Representations

3.1.2 Why Embeddings Enable Similarity Search

3.1.3 The many ways to create an Embedding

3.2 Collaborative filtering (CF)

3.3 Collaborative Filtering with Matrix Factorization

3.3.1 How Matrix Factorization Works

3.3.2 From SVD to Alternating Least Squares

3.3.3 Implementing ALS

3.3.4 Training an ALS Model

3.3.5 Generating Recommendations

3.3.6 Strategies for multiple seeds

3.3.7 Using the framework

3.3.8 Choosing Hyperparameters

3.3.9 Production Considerations

3.4 Embeddings from Content: BERT

3.4.1 Sentence Transformers: BERT for Similarity Search

3.4.2 Creating Content Embeddings

3.4.3 Finding Similar Items

3.5 Embeddings from Language Models

3.5.1 What Makes LLM Embeddings Different?

3.5.2 Using Open-Source LLM Embeddings

3.5.3 Steering Embeddings with Instructions

3.5.4 When LLM Embeddings Provide Real Value

3.6 Combining Multiple Embedding Sources

3.7 Summary