liveBook · Manning

Candidate sourcing

As explained in Hidden Influences

A recommender-system stage that selects a relatively small set of potentially relevant items from a larger item collection.

When you open a social media feed, the platform cannot practically rank every post ever made before showing you something. Candidate sourcing, also called candidate generation, first selects a smaller pool of potentially relevant items for later ranking. section 2.2 section 2.2.1

This stage is the start of a funnel: an item left out cannot be ranked or shown later. The aim is not to put items in a final order, but to discard as few potentially interesting items as possible while keeping the pool manageable. Candidate generation and filtering can reduce thousands or millions of items to a few hundred. section 2.2.1 section 2.8

Possible ways to find candidates include:

  • For a streaming service, consider items in genres the user has consumed before. section 2.2.1
  • For a social feed, consider recent posts from followed users or topics, or a post liked by many people the user follows—even if its creator is not followed. section 2.2.1

Filtering can then remove unwanted items, such as blocked content. Heavy ranking orders the remaining candidates by predicted value, and optional post-processing can adjust that order before delivery. These later steps can reorder or remove candidates, but they cannot make an omitted item appear unless it is added to the pool. section 2.2 section 2.2.1

Figure 2.1 A bird’s-eye overview of a recommender system. Recommendations are not the output of a single operation; rather, they are the results of multiple interlinked operations. Candidates are generated from the universe of all possible content—for example, by picking posts from users that are followed. Later, candidates are filtered to restrict the pool to a more manageable number. At this stage, the content gets actively sorted by the Heavy Ranker in accordance with a predefined metric. Such a ranked list can be adjusted, if needed, before being delivered to the users, whose engagement is recorded. Platform policies influence the filtering and post-processing steps, for example, by limiting the amount of content from the same author.

Follow candidates from the content pool through filtering, ranking, delivery, and recorded engagement. section 2.2

How Hidden Influences covers it

How deep it goes
Explains the selection trade-off and possible candidate sources, without implementation details.
Examples
Uses streaming genres, recent followed posts, and posts liked by people a user follows as candidate sources.
What you can do after
Evaluate candidate selection by considering speed and the risk of excluding potentially interesting items.
How it connects
Connects candidate sourcing to filtering, ranking, post-processing, and user feedback.

Also explained in

Compare how 3 books explain candidate sourcing →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Hidden Influences. The section links point to the parts of the book it draws on.