Cost optimization reduces the money or resources needed to operate an LLM application while preserving its required behavior. LLM charges depend on input and output tokens, and model pricing varies. section 9.6.2
Practical ways to reduce LLM costs include:
- Keep prompts short and limit generated output with concise instructions or a maximum token value. section 9.6.2
- Batch requests when possible instead of sending repeated prompts separately. section 9.6.2
- Use cheaper models for less complex tasks and check current provider pricing. section 9.6.2
- Ask users to provide their own API keys, recognizing the resulting account-creation burden. section 9.6.2
Trivia-game requests
A trivia game can reduce costs by generating several questions in one request rather than repeatedly sending the list of previously asked questions. Use a tokenizer to estimate prompt and response costs before choosing the approach. section 9.6.2
