Image search retrieves images or related content from visual or textual queries. Traditional search matches labels or text descriptions in an inverted index; dense vector search encodes images as embeddings and retrieves visually or conceptually similar images. section 15.3.1
A vision Transformer can encode each image into a vector for indexing, then encode an input image and search for nearby vectors. Training images alongside descriptions can place image and text representations in one shared vector space, enabling text-to-image and image-to-image search. section 15.3.1 section 15.3.1
Query modes
A user can submit an image to find visually similar images, or enter descriptive text to retrieve images whose visual features match beyond their captions. CNNs or vision Transformers can provide the representations. section 15.3.1
