Audio search retrieves audio content, or content characterized by sound, in response to textual, spoken, or audio queries. Traditional systems transcribed audio into text, indexed the transcription, converted spoken queries into text, and searched the text index. section 15.3.1
The approach is expanding beyond spoken words. Multimodal Transformer models can map text and audio into overlapping vector spaces, allowing search to represent qualities such as environmental sounds, voices, and accents directly. section 15.3.1
Audio search supports queries such as:
- “loud train,” targeting an environmental sound. section 15.3.1
- “rushing stream,” targeting a nonverbal audio event. section 15.3.1
- “Irish accent,” targeting a characteristic of speech rather than specific words. section 15.3.1
