Video search retrieves scenes by matching text descriptions, audio clips, or images against indexed video representations. Because video combines sequential images and audio, search can use either modality or both. section 15.3.1
A practical pipeline maps video frames, audio, and written text into overlapping vector spaces. It indexes every frame or samples a few frames per second, depending on the required search granularity, then compares queries with those representations. section 15.3.1
Multimodal scene retrieval
A text query can locate a scene matching its description; an audio clip can retrieve related video; an image can find the most similar video. Audio-aware search can also target sounds such as a loud train or rushing stream. section 15.3.1
