A digital image is unstructured visual data represented as a grid of colors that forms a picture. Search systems handle images alongside text, audio, and video. section 2.1.1
Image search can represent and retrieve images in two main ways:
- Search an inverted index using text descriptions or labels attached to images. section 15.3.1
- Encode images as embedding vectors, index those vectors, and encode a query image to retrieve visually or conceptually similar images. section 15.3.1
From picture to search result
A stored image can carry a text label for inverted-index search. Alternatively, a vision Transformer can convert the stored image and a new query image into embedding vectors, allowing the system to rank visually similar images. section 15.3.1
Figure 2.2 Multiple types of unstructured data. In addition to the text from the last figure, we now see images, audio, and video, which are other forms of unstructured data.
Use the figure to compare images with other unstructured data types, including text, audio, and video. section 2.1.1
