If you're choosing an LLM for a production SaaS in 2026, the short answer is: it depends on whether you're optimising for reasoning, cost, or latency. The long answer involves context windows, tool-use reliability, cache behaviour, and a few places the newer models are still catching up. Here's the decision framework we actually use when wiring an LLM into a client's backend — tradeoffs, real numbers where we have them, and the mistakes we made learning this the hard way.
The field, in one table
Three flagship tiers from three vendors dominate production SaaS work today. They're close enough on most tasks that headline benchmarks don't tell you much — the real differences show up at scale, under load, and when things go wrong.
| Model | Context window | Output max | Best at | Watch out for |
|---|---|---|---|---|
| Claude Opus 4.7 | 1M tokens | 128K tokens | Coding, long-context reasoning, agentic tool use | Flagship-tier pricing |
| Claude Sonnet 4.6 | 1M tokens | 64K tokens | Most production workloads — balanced cost/quality | Occasionally slower first-token latency |
| GPT-5 | 400K tokens | 128K tokens | Creative writing, general reasoning, wide tool ecosystem | Rate limits bite earlier at scale |
| Gemini 3 Pro | 2M tokens | 64K tokens | Massive context, multimodal, low cost | Tool-use reliability lags Anthropic |
Treat this table as a snapshot. Pricing and feature ceilings change every quarter, and the right answer for your workload six months from now may differ from the right answer today. The framework below matters more than the exact numbers.
Cost — where the decision usually gets made
Sticker pricing (dollars per million input and output tokens) is the easy comparison everyone does first. At production volume, three other factors matter more: cache hit rate, batch discounts, and the ratio of input to output tokens in your actual workload.
- Prompt caching reliably drops costs 50–90% on RAG and tool-use workloads where the system prompt and retrieved context are stable across turns. Anthropic's caching has been the most mature in production; Google and OpenAI closed most of the gap through 2025.
- Batch APIs cut costs roughly in half for any workload where latency doesn't matter — overnight embeddings, bulk classification, backfills.
- Output tokens are 3–5× the cost of input. A chatbot that outputs long responses burns budget far faster than one that outputs structured JSON.
A worked example
Say you're running a customer-support assistant: 10,000 conversations a day, 15 average turns per conversation, a 20K-token knowledge base loaded as context, and 150 tokens of output per turn. That's 150,000 turns per day. Without prompt caching, you pay for 20K input tokens per turn — three billion input tokens per day. With caching, cache hits cost roughly 10% of a fresh read. The difference between the two monthly bills is the difference between a viable product and a burning one.
// Anthropic prompt caching — the boring but important pattern
const response = await anthropic.messages.create({
model: "claude-sonnet-4-6",
max_tokens: 1024,
system: [
{ type: "text", text: SYSTEM_PROMPT, cache_control: { type: "ephemeral" } },
{ type: "text", text: knowledgeBase, cache_control: { type: "ephemeral" } },
],
messages: [
{ role: "user", content: userTurn },
],
});
// Second call within 5 minutes → system + knowledgeBase come from cacheTool use — where Anthropic still leads
Agentic workflows are the dividing line in 2026. Models that reliably call your tools in the right order, handle tool errors gracefully, and know when to stop are worth their weight in GPU hours. On complex multi-tool workflows — say, an AI that reads an inbox, books a meeting, and drafts a confirmation email — Claude's tool-use reliability continues to outpace the competition. GPT-5 closes the gap for single-tool calls but still occasionally over-invokes tools in multi-step chains. Gemini 3 is the newest to serious tool-use and it shows.
If your product depends on the model orchestrating real side effects (payments, sends, writes to a DB), test it on Claude Sonnet first. If it works there, you have a working baseline; if it doesn't, no other model will save you.
Latency and throughput
Latency matters most for interactive UX; throughput matters for agents and batch jobs. The ranking flips depending on which you're optimising for.
- For streaming chat: GPT-5 and Gemini 3 Flash have the edge on time-to-first-token — usually 300–700ms — which feels meaningfully snappier in a chat UI.
- For agent turns: Claude's throughput advantage on complex reasoning usually wins — fewer retries, fewer re-prompts, fewer turns to complete a task.
- Regional availability matters. If your users are in APAC or Europe, check which region your preferred model actually serves from before committing.
Context windows — 1M–2M tokens is not a free lunch
Large context windows are a capability, not a strategy. Loading a 500K-token document into every turn will work — and it will also cost you five times as much as a smart retrieval pipeline. The right pattern for most SaaS apps is still: RAG for dynamic content, caching for the stable prefix, and long context only when the task genuinely spans the whole document.
Attention quality degrades somewhere past 200K tokens on every model we've tested. If accuracy matters, chunk and retrieve — don't just stuff the window.
The decision framework
Ignore the benchmark-of-the-week and pick on the four axes that actually drive production spend and user experience:
- Workload shape — is this interactive chat, async agents, structured extraction, or batch classification? Each has a different winner.
- Tool complexity — how many tools does the model need to orchestrate? More than three, default to Claude.
- Budget ceiling — set a monthly cost ceiling before you start. Pick the cheapest model that meets your quality bar, not the best one you can technically afford.
- Vendor concentration risk — if you're building a product where the LLM is the product, wire up a fallback model behind a router. Outages happen and rate limits bite unexpectedly.
What we'd pick today for a new SaaS
For a greenfield SaaS product starting today, our default stack is: Claude Sonnet 4.6 as the primary model for user-facing work, Claude Haiku 4.5 for classification and routing, and either GPT-5 or Gemini 3 Pro as a failover. Sonnet hits the cost/quality sweet spot for 95% of SaaS workloads. Haiku is cheap and fast enough to pre-classify intents before routing to the heavier model. And having a second vendor wired in behind a feature flag means you're never one incident away from a product outage.
The model you pick on day one is rarely the model you'll be running in production six months later. Build your code to treat the LLM as a swappable dependency — a thin adapter layer, not baked-in SDK calls — and you'll thank yourself the first time pricing changes or a better model ships.
Key takeaways
- Benchmarks lie. The right model is a function of your workload, your tool complexity, and your cost ceiling — not the top line of a leaderboard.
- Prompt caching is a 5–10× cost lever on RAG and long-context workloads. If you're not using it, that's the biggest optimisation available.
- Tool-use reliability is where Anthropic still earns its premium — and where agentic products live or die.
- Long context windows are a capability, not a solution. RAG plus caching beats stuffing the window on most workloads.
- Build for model swappability from day one. The model is a dependency; treat it like one.