A search query is the main object handled by a search engine. Its quality affects both validation design and production data health: real user queries must be represented without allowing duplicate or leaked examples to distort evaluation. section 7.5.2 section 14.5.2
Build reliable query datasets by following these checks:
- Represent real production queries and cover varied topics and contexts. section 7.5.2
- Group queries by user, remove duplicates, and keep each user in only one split. section 7.5.2
- Filter obvious garbage while retaining plausible unusual queries to avoid training-serving skew. section 14.5.2
- Monitor query-related search stages such as candidate fetching, final ranking, and override application. section 14.5.2
def assign_bucket(user_id):
_hash = sha1(user_id.encode()).hexdigest()
return int(_hash, 16) % n_buckets
def assign_split(user_id):
bucket = assign_bucket(user_id)
if bucket < n_buckets * train_ratio:
return 'train'
elif bucket < n_buckets * (train_ratio + val_ratio):
return 'val'
else:
return 'test'
Use deterministic user-ID hashing to assign stable training, validation, and test splits while allowing future ratio changes. section 7.5.2
