Why cost control gets ignored
LLM bills usually start quietly. A prototype turns into a feature, the feature becomes traffic, and suddenly you are paying for verbose outputs, duplicate requests, and context bloat. By 2026, the teams that stay profitable are the ones that treat token spend like bandwidth: measured, routed, and trimmed early.
I have seen a simple prompt pipeline cut monthly spend by more than half after the team added caching and stopped sending entire chat histories into every request. The model did not get worse. The prompt got smarter.
Who needs this
This is for teams running AI features in production, especially if you're paying for repeated summaries, support answers, or coding helpers. It is also useful if you are pre-PMF and every dollar matters.
The four levers that matter
The easiest way to save money is to attack spend from four sides:
- Reduce input tokens.
- Reduce output tokens.
- Cache repeated work.
- Route cheap tasks to cheaper models.
That sounds obvious. Most teams still skip at least two of those.
Input trimming
Do not feed the model the full conversation if only the last two turns matter. Do not include huge logs when a short summary will do. And do not send metadata the model n the model never uses.
ever uses.
ever uses.
ts
function compactMessages(messages: { role: string; content: string }[]) {
return messages.slice(-6).map(m => ({ role: m.role, content: m.content }))
}
This one line can save real money if your app is chat-heavy.
Output control
The cheapest token is the one the model never generates. Set response limits. Ask for bullets instead of essays. Force JSON when the UI only needs structured data.
ts
const response = await client.responses.create({
model: 'gpt-5-mini',
input: 'Summarize this in 5 bullets.',
max_output_tokens: 180
})
If your app doesn’t need poetry, do not pay for poetry.
Caching that actually helps
AI SDK caching guidance in 2026 points toward caching final responses and using stream-aware approaches when you want to preserve the user experience
. That matters because users do not care whether the model regenerated the same answer again. They care that it felt instant.
Best candidates for caching:
- FAQ answers.
- Repeated code explanations.
- Document summaries.
- Prompt templates with stable inputs.
Worst candidates:
- Highly personalized replies.
- Fast-changing data.
- Any request that depends on live state.
Route by task, not model brand
A good savings strategy is to separate tasks by complexity.
| Task type | Best model tier | Reason |
|---|
| Classification | Cheap mini model | Output is tiny |
| Extraction | Cheap/fast model | Low reasoning load |
| Drafting | Mid-tier model | Balanced quality |
| Hard reasoning | Premium model | Worth the spend |
That kind of routing often delivers the biggest savings with the least user-visible risk.
Prompt compression
Prompt compression is just a fancy way to say: write better prompts. Remove repeated instructions. Replace paragraphs with rules. Move stable policy text into a system message. Use shorter exemplars.
A compact prompt is not only cheaper; it is often more reliable because the model has less noise to ignore.
Practical failure modes
The biggest failure mode is over-optimizing one path while ignoring the rest. You save 20% on prompt tokens and then waste 60% on repeated retries or oversized outputs. Another failure mode is using the same premium model for all traffic because nobody wanted to build routing.
That is the expensive choice.
What I’d ship first
If you want a quick win, do these three things first:
- Add response caching for repeated prompts.
- Cap output tokens.
- Route low-complexity requests to a cheaper model.
Those changes are boring, but they work. In production, boring is usually profitable.
Conclusion
You do not need a miracle to cut LLM costs. You need discipline. Trim inputs, cap outputs, cache aggressively, and stop sending premium models every request that smells complicated.
If you do that well, a 60% reduction is not crazy. It is often the result of basic engineering that most teams postpone until the bill forces the issue.