ai-adoption · saas · cost-optimisation · prompt-caching · business-case
Cost Optimisation: some models are just cheaper
A £3 budget alert sent me down the AI-model-pricing rabbit hole. What I found - sneaky caching costs, order-of-magnitude spread between models, and why the answer for most SaaS chat workloads is boring.
The details of this report are unashamedly generated by AI to support me doing some business case work for a particular SaaS product.
I was in the process of building out a Software as a Service product that makes use of embedded AI to support data capture and decision making and came across a challenge.
One of my code changes had caused it to start to consume my API credits on Anthropic. I keep these pretty lean as I am on a budget, so when my alert went off that I had gone over my budget (a modest £3, yes I know I am tight,) I realised I was missing some important metrics. So I decided to dig.
To my surprise I found that there was a huge disparity across the range of models, and being cost conscientious, realised that there are some really cheap and effective models out there.
To save you the pain of reading this whole rather technical document here is the summary:
- Context caching via API is a great way to save some token usage and greatly encouraged. (Remembering that you cant really cache different users configurations and content)
- There are sneaky costs to content caching that you need to watch out for and you will get charged for them.
- At the moment Gemini 2.5 flash is the cheapest well known model that is out there.
- The use of the per turn phrase is a standard nomenclature for a single LLM API round trip.
- If you don't cache some of the content for these round trips it becomes cumulatively expensive.
- A lot of the models have implicit caching (it tries its best to cache).
- Explicit caching can help your API calls be more cost effective.
I do encourage you to peruse the rest of the article.
The analysis covers prompt caching architecture, and covers the state of the market as of July 2026.
1. Validation of the Original Table
The formula used in the original comparison - (input_tokens / 1M × input_$) + (output_tokens / 1M × output_$) - is arithmetically correct.
The output figures were correct for the input prices used, but the base input price for Gemini 2.5 Flash was stale ($0.075/M reflected an earlier pricing tier that Google retired). Current published rate is $0.30/M input · $2.50/M output. Every row using Gemini 2.5 Flash in downstream tables reflects the corrected figure.
Benchmark turn: 5,000 input + 500 output tokens (typical Overture planner chat turn from live measurement).
Math check (Gemini 2.5 Pro, unchanged):
Cost = (5,000 / 1,000,000 × $1.25) + (500 / 1,000,000 × $5.00)
= $0.00625 + $0.00250
= $0.00875
$0.00875 × 1,000 turns = $8.75 ✓
2. Context Caching Architecture
The multi-tenant SaaS cache-split pattern used in Overture:
┌─────────────────────────────────────┐
│ FIXED (cache_control: ephemeral) │ ← Cached: SAFE to share across tenants
│ • Base persona │
│ • Response protocol │
│ • Section markers rules │
│ • PLAN_SECTIONS list │
│ • Voice guardrails │
├─────────────────────────────────────┤ ← Cache breakpoint
│ TENANT-SPECIFIC (never cached) │ ← Fresh every call
│ • your session specific data │
└─────────────────────────────────────┘
Why this ordering is right
- Prefix alignment. Caches key on exact byte-for-byte prefix. Putting shared static content first guarantees a warm cache across every request from any tenant.
- Tenant isolation invariant. With a single shared API key across all customers, cached content is content-addressed - it doesn't leak between tenants unless two tenants generate identical prefixes. Everything above the breakpoint is generic; nothing below it is cached. Cross-tenant contamination is architecturally impossible.
- Dynamic-first-tail rule. The most-variable content (user message, current turn state) belongs at the absolute tail, so a change there never invalidates the cache. Our layout complies.
3. Prompt Caching Mechanics by Provider
| Feature | Anthropic (Claude) | Google Gemini 2.5 | OpenAI (GPT-4o) | DeepSeek (unverified) |
|---|---|---|---|---|
| Trigger mechanism | Explicit (cache_control block) | Explicit (Context Caching API) + implicit auto-caching on 2.5 series | Automatic (prefix-matching) | Automatic (prefix-matching) |
| Minimum cache size | Per model - see below | Not publicly specified on 2.5-flash | 1,024 tokens | 1,024 tokens |
| Cache read discount | 90% off input rate (0.1× base) | 90% off input rate ($0.03/M vs $0.30/M base on Flash) | 50% off input rate | ~90%+ off (varies) |
| Cache write cost | +25% on 5-min write / +100% on 1-hour | Storage-only (no write premium) | Free | Free |
| Storage cost | None | $1/M tokens/hour (2.5 Flash) · $4.50/M/hr (2.5 Pro) | None | None |
| Cache TTL | 5 min (ephemeral) or 1 hour | Configurable; 5-min default in the SDK we use | ~5-10 min | ~10 min |
Anthropic - verified per-model cache minimums (from platform.claude.com/docs)
| Model | Minimum cacheable prompt |
|---|---|
| Claude Fable 5, Mythos 5 | 512 tokens |
| Claude Opus 4.8, Sonnet 5, Sonnet 4.6, Sonnet 4.5, Haiku 4.5 | 1,024 tokens |
| Claude Opus 4.7 | 2,048 tokens |
| Claude Opus 4.6, Opus 4.5 | 4,096 tokens |
| Claude Haiku 3.5 | 2,048 tokens |
Prompts below the minimum are silently processed without caching - no error is returned. Overture's cached prefix is ~1,400 tokens, above Haiku 4.5's 1,024-token floor, so our caching works. If we drop the prefix below 1,024 tokens in a future revision, caching will silently no-op - worth a regression test.
Gemini - implicit caching on 2.5 series
Google added implicit prompt caching for 2.5-flash and 2.5-pro in early 2025 - identical prompt prefixes across requests are automatically discounted with no storage fee and no explicit API call. This is why Overture's gemini.ts no longer uses GoogleAICacheManager - the implicit path handles the fixed-prefix discount for free, and the explicit path is a net cost at our current volume because of the $1/M/hour storage fee.
Gemini - break-even for explicit caching (2.5 Flash)
At corrected pricing ($0.30/M base, $0.03/M cache read):
- Savings per read: 1,400 tokens × ($0.30 − $0.03) / 1M = $0.000378
- Storage cost per hour: 1,400 × $1 / 1M = $0.0014
- Reads needed per hour to break even: ~4
At even modest traffic, Gemini explicit caching is profitable. Our earlier analysis using the stale $0.075/M base rate showed break-even at ~18 reads/hour, which was misleading. Nevertheless, implicit caching dominates - same discount, zero storage fee, zero code to manage.
4. Expanded Model Pricing Table (Standard Turn)
Benchmark: 5,000 input + 500 output tokens · standard (uncached) rates.
[!NOTE] Tokenizer footnote. Claude Fable 5, Mythos 5, Opus 4.7 and later Opus, Sonnet 5 use a newer Anthropic tokenizer that produces ~30% more tokens for the same English text than earlier Claude models. On a like-for-like text basis, real-world cost for these models is proportionally higher than the naive per-token table below suggests.
| Model | Input ($/M) | Output ($/M) | Per turn | Per 1,000 turns | Status |
|---|---|---|---|---|---|
| Claude Haiku 4.5 | $1.000 | $5.00 | $0.00750 | $7.50 | Verified · Anthropic docs |
| Gemini 2.5 Flash ¹ | $0.300 | $2.50 | $0.00275 | $2.75 | Verified · ai.google.dev |
| Gemini 2.5 Pro | $1.250 | $5.00 | $0.00875 | $8.75 | Verified · ai.google.dev |
| Claude Sonnet 5 (intro through 31 Aug 2026) ² | $2.000 | $10.00 | $0.01500 | $15.00 | Verified · Anthropic docs |
| Claude Sonnet 5 (from 1 Sep 2026) ² | $3.000 | $15.00 | $0.02250 | $22.50 | Verified · Anthropic docs |
| Claude Sonnet 4.6 | $3.000 | $15.00 | $0.02250 | $22.50 | Verified · Anthropic docs |
| Claude Opus 4.7 / 4.8 ² | $5.000 | $25.00 | $0.03750 | $37.50 | Verified · Anthropic docs |
| Claude Fable 5 ² | $10.000 | $50.00 | $0.07500 | $75.00 | Verified · Anthropic docs |
| Claude Mythos 5 ² | $10.000 | $50.00 | $0.07500 | $75.00 | Verified · Anthropic docs · limited availability |
| OpenAI GPT-4o-mini | $0.150 | $0.60 | $0.00105 | $1.05 | Verified (Jan 2026) |
| OpenAI GPT-4o | $2.500 | $10.00 | $0.01750 | $17.50 | Verified (Jan 2026) |
| Gemini 3.5 Flash (unverified) | $1.500 | $9.00 | $0.01200 | $12.00 | Cannot confirm model exists |
| Gemini 3.1 Pro (unverified) | $2.000 | $12.00 | $0.01600 | $16.00 | Cannot confirm model exists |
| DeepSeek-V4-Flash (unverified) | $0.140 | $0.28 | $0.00084 | $0.84 | Rates change frequently |
| DeepSeek-V4-Pro (unverified) | $0.435 | $0.87 | $0.00261 | $2.61 | Rates change frequently |
¹ Gemini 2.5 Flash base price corrected from stale $0.075/M. Google published $0.30/M for text input (paid tier) as of July 2026.
² Uses the newer Anthropic tokenizer - see tokenizer footnote above.
5. Pricing with Caching Enabled (4,500 cached / 500 fresh input / 500 output)
[!TIP] Cache-hit steady state assumed - the cache-write cost (Anthropic 1.25× first turn) is amortised across many reads. Storage cost included where applicable.
| Model | Cache hit rate | Cache read $/M | Cached input | Fresh input | Output | Storage/hr* | Per turn | Per 1,000 turns |
|---|---|---|---|---|---|---|---|---|
| Claude Haiku 4.5 | 90% off | $0.100 | $0.00045 | $0.00050 | $0.00250 | - | $0.00345 | $3.45 |
| Gemini 2.5 Flash (implicit) | 90% off | $0.030 | $0.000135 | $0.00015 | $0.00125 | - | $0.00154 | $1.54 |
| Gemini 2.5 Flash (explicit) | 90% off | $0.030 | $0.000135 | $0.00015 | $0.00125 | $0.0014 | see note † | see note † |
| Gemini 2.5 Pro | 90% off | $0.125 | $0.000563 | $0.000625 | $0.00250 | $0.0063 | $0.00369 + storage | Traffic-dependent |
| Claude Sonnet 5 (intro) | 90% off | $0.200 | $0.00090 | $0.00100 | $0.00500 | - | $0.00690 | $6.90 |
| Claude Sonnet 5 (standard) | 90% off | $0.300 | $0.00135 | $0.00150 | $0.00750 | - | $0.01035 | $10.35 |
| Claude Opus 4.7 / 4.8 | 90% off | $0.500 | $0.00225 | $0.00250 | $0.01250 | - | $0.01725 | $17.25 |
| Claude Fable 5 | 90% off | $1.000 | $0.00450 | $0.00500 | $0.02500 | - | $0.03450 | $34.50 |
| Claude Mythos 5 | 90% off | $1.000 | $0.00450 | $0.00500 | $0.02500 | - | $0.03450 | $34.50 |
| OpenAI GPT-4o-mini | 50% off | $0.075 | $0.00034 | $0.00008 | $0.00030 | - | $0.00072 | $0.72 |
| OpenAI GPT-4o | 50% off | $1.250 | $0.00563 | $0.00125 | $0.00500 | - | $0.01188 | $11.88 |
| DeepSeek-V4-Flash (unverified) | ~90% off | $0.014 | $0.00006 | $0.00007 | $0.00014 | - | $0.00027 | $0.27 |
| DeepSeek-V4-Pro (unverified) | ~90% off | $0.044 | $0.00020 | $0.00022 | $0.00044 | - | $0.00086 | $0.86 |
* Gemini storage cost is per hour of cache lifetime, independent of read count. Amortised across reads. † Gemini 2.5 Flash explicit-cache cost per 1,000 turns depends on read rate. At 1 turn/hour storage dominates and cached turns cost more than uncached; at 100 turns/hour storage is negligible. Implicit caching has no storage fee and is the operationally correct choice for 2.5-flash.