Top-k limits a choice to a fixed number of highest-probability candidates. In language generation, it narrows the tokens eligible for random selection; in document retrieval, it specifies how many similar documents to return. section 3.4.1 section 4.4.1
Use top-k to control the size of a candidate set:
- Set generation top-k alongside temperature or Top-P when configuring ChatOptions, either globally in application.properties or programmatically. section 3.4.1
- Set retrieval top-k on SearchRequest to return at most the requested number of most-similar documents. section 4.4.1
- Balance retrieval coverage against prompt size: larger values add context and token cost, while smaller values may omit information needed for a correct answer. section 4.4.1
Spring AI configuration
Configure generation with spring.ai.ollama.chat.options.top-k=4 or ChatOptions.builder().topK(4).build(). Configure retrieval with SearchRequest.builder().query(question).topK(6).build(), optionally combining it with metadata filtering and a similarity threshold. section 3.4.1 section 4.4.1
ChatOptions chatOptions = ChatOptions.builder()
.topK(4)
.build();
Shows the programmatic ChatOptions setting for limiting generation candidates. section 3.4.1
var searchRequest = SearchRequest.builder()
.query(question)
.topK(6)
.filterExpression(
new FilterExpressionBuilder()
.eq("gameTitle", normalizeGameTitle(gameName)).build())
.build();
Shows a retrieval request returning at most six documents while filtering results by game metadata. section 4.4.1
