Similarity search finds data whose meaning is close to a query, even when the words do not match. In vector search, an embedding model turns content and queries into numerical vectors. Nearby vectors indicate greater similarity. A search for “renewable energy,” for example, can retrieve material about solar power or wind energy without requiring that exact phrase. section 12.0 section 12.1.2
Figure 12.2 Vector database operations. Data is processed by an embedding model, transforming it into vector embeddings such as [1.45, ..., 1.01, 0.89, 0.06]. These embeddings are stored in a vector database. Queries from the application are also transformed into embeddings before the database is searched. The most relevant results based on these query embeddings are then retrieved and returned as query results. (Image © MongoDB 2024 CC BY-NC-SA 3.0)
Follow the two paths: content becomes stored embeddings, while a query becomes an embedding used to retrieve the closest results. section 12.1.2
To put similarity search to work in Atlas:
- Generate embeddings for the content you want to search, and store the resulting vectors with the documents. Convert incoming queries with the same or a similar embedding model. section 12.1.2
- Create an Atlas Vector Search index that identifies the embedding field. Choose vector dimensions with the use case and computational cost in mind; supported embeddings can have up to 4,096 dimensions. section 12.2.1
- Query the indexed vectors with the $vectorSearch aggregation pipeline stage. An application can submit a query vector, identify the vector field, and request a number of results through a MongoDB driver. section 1.5 section 12.4.1
- Where appropriate, narrow candidates with prefilters, such as a category and price range, before comparing vectors. section 12.4.2
Closeness can be assessed with measures such as cosine similarity or Euclidean distance. Unlike exact-text matching, this approach can surface conceptually related results; it also depends on embeddings that represent the content and on an index for efficient retrieval. Atlas Vector Search uses Apache Lucene to index and navigate high-dimensional vectors. section 12.1.2 section 12.0
Grounding a chatbot response
Split documents into chunks, embed and store them, then embed a user’s question. Similarity search retrieves relevant chunks; placing those chunks in the language model’s prompt gives its response context from the stored documents. This is the retrieval step in retrieval-augmented generation (RAG). section 14.1
