Top-p filtering limits generation to the smallest set of candidate tokens whose cumulative probability reaches the configured value, from 0 to 1. section 3.4.1
Configure top-p in Spring AI
Set top-p globally with spring.ai.openai.chat.options.top-p=0.8, or configure it programmatically with ChatOptions.builder().topP(0.8).build(). section 3.4.1
Top-p works alongside other chat options:
- Temperature scales candidate probabilities; top-p and top-k eliminate lower-ranked candidates. section 3.4.1
- A lower top-p value retains a smaller probability set, while a higher value allows more candidates. section 3.4.1
spring.ai.openai.chat.options.top-p=0.8
Use the Spring Boot property listing to configure top-p for an OpenAI chat model. section 3.4.1
ChatOptions chatOptions = ChatOptions.builder()
.topP(0.8)
.build();
Use the ChatOptions listing to set top-p programmatically. section 3.4.1
