Cost is the total price of obtaining an output, not just its token bill. It includes retries, human review, waiting, infrastructure, and the incidents or defects the output causes. A cheaper model can cost more per shipped feature if it produces slower or less reliable results. section 1.3 section 9.4.1
Cost competes with accuracy, safety, creativity, and latency. More context or retrieval may improve results but add expense and failure surfaces. Cutting context, choosing a cheaper model, or skipping verification can save tokens while shifting expense to people who must diagnose and repair the result. section 1.3 section 9.0
Build a cost picture around the work delivered, rather than the invoice alone:
- Count subscriptions, API usage and overages, CI runners, sandboxes, and other supporting compute. section 9.2.1 section 10.3.6
- Estimate review, waiting, context switching, retries, and downstream bugs. State assumptions: a wait-time estimate based on single-task work can be an upper bound when developers can parallelize. section 9.4.1 section 10.3.6
- Measure quality and delivery alongside spending. Cost per developer alone does not establish what the investment produces. section 9.4 section 9.2.7
Pricing context in text-to-SQL
A benchmark of 316 queries compared different numbers of recent queries in the prompt. Ten examples cost 7 cents per benchmark run; 20 cost 10 cents, while simple-query p50 accuracy rose from about 0.64 to 0.71. At a steady 100 requests per second, scaling the 7-cent run gives approximately $1,914 per day. Five examples cost 6 cents per run, trading some accuracy for lower spend. These are benchmark-specific choices, not universal context limits. section 6.4.2
Optimize by workflow. Use a capable model where complex reasoning warrants it, a cheaper option for simple work, and asynchronous processing where nobody needs an immediate answer. For agents, constrain relevant files, output, and iteration depth, but retain the tests needed to verify the change. section 9.5.4 section 9.5.2
A retry loop can dominate the bill
An agent changing a date format encountered a flaky integration test, inferred the wrong cause, and repeatedly altered retry logic. Forty-seven iterations consumed $30 in tokens and introduced three bugs. The token charge was only part of the cost: a human still had to reconstruct and clean up the changes. section 10.4
Table 9.4 Cumulative optimization impact
|
Problem Solved
|
Monthly Cost
|
Savings
|
Effort
|
|
Baseline (Sonnet everything)
|
$8,754
|
-
|
-
|
|
Wrong tool for the task
|
$5,100
|
−$3,654
|
30-min meeting
|
|
Context waste
|
$4,200
|
−$900
|
2-hour audit
|
|
One model for everything
|
$3,300
|
−$900
|
1 afternoon
|
|
Output verbosity
|
$2,500
|
−$800
|
1 hour
|
|
Everything synchronous
|
$1,900
|
−$600
|
1 day
|
|
One tool for all workflows
|
$1,407
|
−$493
|
1 week
|
|
Total reduction
|
$8,754
→ $1,407
|
84%
|
~2 weeks
|
Compare cumulative savings with the effort required at each optimization step; later savings can take substantially more work. section 9.2.8
Stop when the expected savings no longer justify the engineering effort. Also watch for rebound: a lower cost per interaction can encourage more interactions, so the total bill may fall less than unit-cost savings suggest. Neither high nor low token consumption proves that useful work was delivered. section 9.2.8 section 9.5.6
















