liveBook · Manning

Candidate sourcing

As explained in AI Recommender Systems

A recommendation-stage technique that produces the set of items eligible for further processing.

Candidate sourcing is the recommendation-stage technique that produces items eligible for further processing. It is the first step in a multi-stage pipeline: source candidates, filter them, score them, then order the results. section 1.6 section 1.0

The source can begin with user history, profile, and context, or with a general signal such as popularity. Personalization may affect which items are sourced, how they are ordered later, or both. section 1.2.1

A simple item-based implementation proceeds as follows:

  • Represent each item by the users who interacted with it, forming an item-oriented view of the user-item matrix. section 2.2.2
  • Compute item-to-item cosine similarities from overlapping user behavior. section 2.2.2
  • Precompute or store similarities, then retrieve the highest-similarity items for a seed item. section 2.2.2
  • Scale large catalogs with approximate nearest neighbor methods instead of constructing a full similarity matrix. section 2.2.2

Retrieving similar movies

Given Back to the Future (1985) as a seed, item similarity retrieves films such as Star Wars: Episode V, Terminator, and Raiders of the Lost Ark. The retrieval stage narrows roughly 59,000 movies to 100 candidates in milliseconds. section 2.2.2

Listing 2.3 Retrieval function
def retrieve_similar_items(movie_id, k=100):
    if movie_id not in movie_to_idx: #A
        return []
    
    movie_idx = movie_to_idx[movie_id]
    similarities = item_similarity[movie_idx].toarray()[0] #B
    
    top_indices = np.argsort(similarities)[-(k+1):-1][::-1] #C
    
    candidates = []
    for idx in top_indices:
        candidates.append({
            'movie_id': idx_to_movie[idx],
            'similarity': float(similarities[idx])
        }) #D
    
    return candidates
 

The retrieval-function listing shows how a seed item is mapped to its top-k similar candidates and how unknown items return an empty result. section 2.2.2

How AI Recommender Systems covers it

How deep it goes
Defines the pipeline role and demonstrates item-similarity retrieval, but defers approximate nearest-neighbor implementation to later treatment.
Examples
Uses personalized Netflix carousels and item-similarity retrieval from Back to the Future to illustrate candidate sourcing.
What it assumes
Assumes user interaction data and basic recommender stages while introducing the retrieval role and item-similarity implementation.
What you can do after
Build it from scratch with a sparse user-item matrix, cosine similarity, and a top-k retrieval function.
How it connects
Connects candidate generation with filtering, scoring, ordering, collaborative filtering, content-based filtering, and cold-start problems.

Also explained in

Compare how 3 books explain candidate sourcing →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of AI Recommender Systems. The section links point to the parts of the book it draws on.