Self-hosting runs a specialized small language model on organization-controlled infrastructure instead of sending every request to an external API. Its central trade-off is fixed GPU cost versus per-request API pricing: whether it saves money depends on request volume. section 12.9
Calculate the break-even request rate and include API escalations in the hosted path’s cost: escalated requests incur both the SLM attempt and the API charge. Then compare latency and task quality, and verify deployment constraints on the real inference engine. section 12.9 section 12.10
Illustrative cost comparison
With a $2.50-per-hour GPU, a $0.004-per-request API, and 2,950 requests per hour, break-even is 625 requests per hour. At that volume, a 4.2% escalation rate yields about $0.001 per request, 74.6% below the API cost. section 12.9
Listing 12.8 The break-even rate and the cost per request
def breakeven_requests_per_hour(gpu_cost_per_hour, api_cost_per_request):
return gpu_cost_per_hour / api_cost_per_request #1
def slm_cost_per_request(gpu_cost_per_hour, requests_per_hour,
api_cost, escalation):
fixed = gpu_cost_per_hour / requests_per_hour #2
return fixed + escalation * api_cost #3
api_cost = 0.004
be = breakeven_requests_per_hour(gpu_cost_per_hour=2.50,
api_cost_per_request=api_cost)
slm = slm_cost_per_request(
gpu_cost_per_hour=2.50, requests_per_hour=2950,
api_cost=api_cost, escalation=0.042)
saving = 1 - slm / api_cost
print(f"break-even at {be:.0f} requests/hour")
print(f"SLM cost per request ${slm:.6f} ({saving:.1%} below the API)")Use the listing to calculate break-even volume and self-hosted cost per request, including escalations. section 12.9








