Token count measures the input and output tokens consumed by a language-model interaction. Providers use tokens to calculate billing, so tracking them helps estimate operating cost and identify opportunities to reduce spending. section 3.5 section 9.1.3
In a Spring AI application, retrieve usage metadata from ChatResponse after calling responseEntity(), then log it or expose it through observability metrics. The reported data may be unavailable for some AI services. section 3.5 section 9.1
Use token count to:
- Separate prompt tokens from generation tokens and inspect total usage. section 9.1.3
- Filter usage by operation, provider, requested model, or response model. section 9.1.3
- Recognize that adding document context increases input tokens, cost, and the risk of exceeding a model's context window. section 3.2
- Replace indiscriminate context with relevant retrieval-augmented generation when large documents threaten token limits. section 3.2
Inspecting usage with Actuator
Querying gen_ai.client.token.usage can report 3,624 tokens overall. Filtering gen_ai.token.type=input returns 1,773 tokens, output returns 39, and total returns 1,812 for the illustrated filtered operation set; additional tags distinguish chat and embedding activity and the models involved.
Listing 3.9 Logging token usage
package com.example.boardgamebuddy;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.ai.chat.client.ChatClient;
import org.springframework.ai.chat.client.ResponseEntity;
import org.springframework.ai.chat.metadata.ChatResponseMetadata;
import org.springframework.ai.chat.metadata.Usage;
import org.springframework.ai.chat.model.ChatResponse;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.core.io.Resource;
import org.springframework.stereotype.Service;
@Service
public class SpringAiBoardGameService implements BoardGameService {
private static final Logger log =
LoggerFactory.getLogger(SpringAiBoardGameService.class); #1
private final ChatClient chatClient;
private final GameRulesService gameRulesService;
public SpringAiBoardGameService(
ChatClient.Builder chatClientBuilder,
GameRulesService gameRulesService) {
this.chatClient = chatClientBuilder.build();
this.gameRulesService = gameRulesService;
}
@Value("classpath:/promptTemplates/systemPromptTemplate.st")
Resource promptTemplate;
@Override
public Answer askQuestion(Question question) {
var gameRules = gameRulesService.getRulesFor(question.gameTitle());
var responseEntity = chatClient.prompt()
.system(systemSpec -> systemSpec
.text(promptTemplate)
.param("gameTitle", question.gameTitle())
.param("rules", gameRules))
.user(question.question())
.call()
.responseEntity(Answer.class); #2
var response = responseEntity.response(); #3
var metadata = response.getMetadata();
logUsage(metadata.getUsage()); #4
return responseEntity.entity();
}
private void logUsage(Usage usage) {
log.info("Token usage: prompt={}, generation={}, total={}",
usage.getPromptTokens(),
usage.getCompletionTokens(),
usage.getTotalTokens());
}
}
Inspect the Java service pattern for retrieving ChatResponse metadata and logging token usage alongside a ChatClient request. section 3.5
