stream() makes ChatClient emit an LLM response incrementally instead of waiting for the complete answer. The operation returns Flux<String>, whose pieces typically arrive one word at a time. section 3.4.3
Use streaming by changing the application at three points:
- Replace call() with stream().content() in the service. section 3.4.3
- Return the resulting Flux from the controller. section 3.4.3
- Configure the endpoint for a streamed response MIME type rather than application/json. section 3.4.3
Listing 3.7 A board game service that streams the response
@Override
public Flux<String> askQuestion(Question question) { #1
var gameRules = gameRulesService.getRulesFor(question.gameTitle());
return chatClient.prompt()
.system(systemSpec -> systemSpec
.text(promptTemplate)
.param("gameTitle", question.gameTitle())
.param("rules", gameRules))
.user(question.question())
.stream() #2
.content();
}
Streams a board-game answer through ChatClient after enriching the prompt with game rules. section 3.4.3
