A search service has two jobs that pull in different directions: return relevant results and return them quickly. A sophisticated model can rank a few thousand documents, but applying it across billions is costly. A two-stage system splits the work: first retrieve a manageable set of candidates with fast filtering, then rank those candidates with a more sophisticated model. section 3.3
The stages can be designed around different constraints:
- Candidate filtering favors speed. A simple test, such as whether a document contains queried words, can narrow the database before expensive ranking. section 3.3
- Ranking favors relevance. Once the candidate set is small, slower ML models become practical. section 3.3
- Filtering can also form a cascade: query language and user location may rule out documents before candidate filtering begins. section 3.3
When to consider adding retrieval
In a photo-search experiment, average search time decreased slightly and search results became more relevant. A retrospective check found that 97% of clicked photos appeared in the new model’s top 1,000 results. That finding suggested a next step: add a retrieval stage, or candidate model, before ranking. It was a reason to investigate the pipeline, not a measurement of a retrieval stage already in place. section 12.4.2
