Cosine similarity compares the direction of two vectors. A higher score, closer to 1, indicates greater similarity. In an application, it can rank stored embeddings against a user’s query embedding; in a test, it can provide a similarity score for generated and expected text instead of requiring an exact match. section 5.4.2 section 5.1.3
For retrieval, embed stored content and the user query with the same model. Compare the query embedding with each stored embedding, keep the highest-scoring match, and return its associated answer. The same loop could retain several close matches instead of only one. section 5.4.3 section 5.4.2
Listing 5.11 Performing similarity search on the embeddings
let maxSimilarity = -1; #1
let mostRelevantQuestion = ""; #2
for (const [question, storedEmbedding] of Object.entries(embeddingDB)) {
const similarity =
cosineSimilarity(queryEmbedding, storedEmbedding); #3
if (similarity > maxSimilarity) { #4
maxSimilarity = similarity;
mostRelevantQuestion = question;
}
}
The JavaScript loop calls cosineSimilarity for each stored question embedding and tracks the highest score. The listing uses the function but does not define its implementation. section 5.4.3
Testing a generated response
A test uses stringComparison.cosine.similarity to compare a mocked AI response with expected text, then asserts that the score exceeds 0.7. The threshold is adjustable. Mocking makes the test faster and more controlled, but does not check how the live model will behave. section 5.1.3
Listing 5.3 Testing semantic similarity scoring
import { generateText } from "ai";
import stringComparison from 'string-comparison' #1
function semanticSimilarity(text1, text2) {
let cos = stringComparison.cosine
return cos.similarity(text1, text2);
}
test("generated text is semantically similar to expected output", async () => {
generateText.mockResolvedValue(
"AI greatly influences modern life in many ways."
);
const expected = "AI has a significant impact on
modern society."; #2
const generated = await generateText(
"Describe the role of AI in today's world."
);
const similarity =
semanticSimilarity(expected, generated); #3
expect(similarity).toBeGreaterThan(0.7); #4
});
The listing shows the text-comparison call and its threshold assertion; it is a separate use from comparing stored embedding vectors for retrieval. section 5.1.3
