Reranking reorders an initial set of retrieved candidates using additional signals and rules. In a recommendation pipeline, it turns a broad candidate set into a shorter, ordered list for the user. The chosen signals affect not just relevance but also which items and user interests get exposure. section 7.2 section 3.3.1
Ranking a movie row
A two-tower candidate generator retrieves a few hundred movies using user embedding similarity and metadata. A gradient-boosted decision tree then combines personalization, popularity, recency, freshness, and business rules to produce a final list of about 20 titles. Tuning those signals changes what users see. section 7.2
A high offline score does not guarantee a good list. In an internal beta, employees compared the new and current movie lists for relevance, diversity, and novelty and flagged odd picks. The same popular title repeatedly appeared at the top across users, pointing to an overweighted popularity feature. section 7.2
Figure 7.1 illustrates an example of a systemic ranking bias caught during internal beta testing: The same popular title (represented as a star) repeatedly appears in the top position of the 'Movies To Watch' row across different users and segments. This pattern, observed before launch, indicates an overweighted popularity feature in the re-ranking stage which is a problem that could influence engagement metrics if not caught before.
The figure shows how an overweighted popularity signal can produce the same top recommendation across users and segments. section 7.2
Make reranking changes observable and validate their product impact:
- Log candidate-generator scores, reranker scores, final blended scores, active component variants, and applied rules. Preserve a decision trace so the final list can be reconstructed and changes in online metrics can be attributed more reliably. section 8.7.1
- Use segment diagnostics to identify a possible intervention. For example, reranking could improve exposure for relevant long-tail films when foreign-language film fans have lower recall. section 3.3.1
- After a change, check downstream outcomes such as click-through rate, watch time, retention, or satisfaction. Depending on the segment, validation may use an A/B test, user research, qualitative review, or monitoring over time. section 3.3.1








