Crowdsourcing obtains information, judgments, or work from many people. In search, it turns distributed human input into evidence about query intent, document quality, and relevance. section 4.0
The input may be explicit, such as document tags, corrections, abuse reports, or Mechanical Turk judgments. It may also be implicit: aggregated clicks reveal which documents users considered relevant for a query, without repeatedly asking customers for help. section 4.0
Search systems can process these signals through several reflected-intelligence approaches:
- Signals boosting uses popular interactions to improve generally relevant results. section 4.0
- Collaborative filtering learns relationships between users, items, attributes, and queries. section 4.2.5
- Learning to rank uses interaction data to learn generalized relevance decisions. section 4.0
From reviews to document quality
Customer reviews can generate product ratings or abuse signals. Sentiment analysis can classify comments as positive, neutral, or negative, allowing the system to boost or penalize corresponding documents. Web links provide another form of crowdsourced intelligence: PageRank measures incoming and outgoing links to estimate authority. section 4.2.6 section 4.2.6
Figure 4.8 Many reflected intelligence models. The leftmost box represents user-to-item similarity for recommendations, the next shows learning specific attribute-based preferences for a user’s profile, the third shows learning item-to-item–based similarity for recommendations, and the rightmost shows learning query-to-item recommendations.
Maps the main reflected-intelligence patterns, including user-to-item, item-to-item, attribute-based, and query-to-item relationships. section 4.2.5
