liveBook · Manning

Chapter 2: Anatomy of a recommender system

What you will learn in chapter 2 of Hidden Influences

This introduction presents the black box model of a recommender system as a way to focus on its input and output.

After this chapter you will be able to

What it builds on

Section by section

Introduction

This introduction presents the black box model of a recommender system as a way to focus on its input and output.

Key terms: black box model, ranked list, recommendations

2.1 The user-and-item world

This section explains recommendation as ordering predefined items for users and distinguishes it from search, generative AI, and the producer and consumer roles users can play.

Key terms: recommender system, pre-specified metric, explicit query, implicit query, predetermined list of items, list-shuffling systems, fabrications, generative AI, user, item

2.2 Parts of a RecSys

This section outlines the interconnected phases that produce recommendations and record user feedback.

Key terms: candidate generation, filtering, ranking, heavy ranking, post-processing, predicted value, delivering, recording users’ behavior, feedback loop

2.2.1 Candidate generation

This section explains how candidate generation selects a manageable pool of potentially relevant content for later ranking.

Key terms: candidate generation, funnel, ranking, in-network, out-of-network, Algorithmic model, Network and Subscription models, trending, feedback loop, ads

2.2.2 Filtering

This section explains how filtering removes unsuitable content before ranking and how imperfect filtering models influence what users see.

Key terms: filtering, ranking, soft-block, terms of service, Not Safe For Work, NSFW, algorithmic models, biased

2.2.3 Heavy ranking

This section explains how heavy ranking predicts different kinds of engagement and combines them using weights that reflect platform priorities.

Key terms: heavy ranking, probability of engagement, Like model, Reply model, rankings, score, coefficient, weight, weights, value model

2.2.4 Post processing

This section explains how post-processing adjusts an already ranked feed before delivery, from spacing content to boosting or downranking particular posts.

Key terms: post-processing, ranked list, intersperse, boosting, ranking score, downrank, shadowbanning

2.2.5 Delivering and recording

This section explains how delivered recommendations receive attention and how users’ reactions are recorded and used to improve recommender systems.

Key terms: users’ reactions, training data, iteration, ranked higher, attention, single session, implicitly learn, interacted with, dwelled on, successful recommendation

2.3 Heavy ranking revisited: Training a RecSys

This section revisits heavy ranking and frames recommender-system training through a library book recommendation example.

Key terms: heavy ranking, scoring, users, items, recommendations

2.3.1 Heuristics

This section explains an interest-and-label-based recommendation approach and its limitations for large collections, user-generated content, and users’ reading histories.

Key terms: content-based approach, labeling, interests, user-generated content, publishers, gatekeepers, previously read item, questionnaire

2.3.2 Data-driven approaches

This section explains how recommender systems use interaction data, collaborative filtering, and optimization metrics to produce and rank recommendations.

Key terms: explicit interactions, implicit signals, machine learning, collaborative filtering, user-item matrix, metadata, metric, success rate, error rate, value

2.4 How to train a machine learning model

This section explains how machine learning uses existing data to discover patterns, using spam filtering and recommendations as examples, and notes its lack of transparency.

Key terms: machine learning, sample data, spam, deep learning, behavioral data, features, weight, transparency

2.5 Raw and derived scores

This section explains raw and derived ranking signals, how models combine and reuse them, and why their dependencies complicate AI governance.

Key terms: signals, features, raw signals, derived signals, toxic score, derived signal, ranking, derived scores, raw features, dependency graph

2.6 Platforms’ design and amplification

This section explains how platform design and multiple interacting features contribute to users’ experiences and content amplification beyond a single recommender system.

Key terms: platforms, recommender, amplification of content, platform design, search functionality, content presentation

2.7 What happens when you refresh your feed

This section explains how embeddings represent users and content and how their distances help generate fast, personalized recommendations.

Key terms: embeddings, vectors, distance, similarity, embedding dimension, cat affinity, dog affinity, embedding dimensions, memory, performance

2.8 Summary

Where this is used later

Concepts explained in this chapter

← Chapter 1: Recommender systems in action Chapter 3: Content moderation →

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.