The LLM bill: one support workload, priced across models, and what actually cuts cost

We re-priced one support-bot workload, token for token, under different models and six cost levers. Most of the savings came from sending fewer tokens and fewer reasoning tokens, not from a cheaper price list.

1. Method

Workload. A fictional SaaS support assistant ("Corvid Cloud support AI", the same workload as the Cost Cutter demo): 37 support questions answered with RAG over a 32-chunk knowledge base (8 are repeats or rephrasings), 24 ticket classifications with gold labels, 12 thread summaries and 8 nightly account digests: 81 requests in total. Monthly figures assume the mix repeats at 1,200,000 requests a month; that volume is an assumption, the per-request token counts are measured.

Runs. Each lever is a real pipeline run through one OpenAI-compatible API gateway (big model claude-sonnet-5-5-high, small model gemini-3.1-flash-lite), temperature 0, on 11 Nov 2026. Token counts come from the API usage field of every call, so hidden reasoning tokens are included the way they are billed. Caching, dedupe and batch are applied to measured tokens: caching with an assumed 95% hit rate on stable prefixes of at least 1,024 tokens, dedupe with an exact plus normalised-text cache replay (3 exact and 5 normalised hits in 81), batch at 50% for the nightly digest only.

Prices. Official list prices read from the providers' pricing pages (Anthropic, OpenAI, Google) on 2026-11-11. As in the Cost Cutter demo, the headline prices each call at the list price of the model that served it: Claude Sonnet 5.5 or Gemini 3.1 Flash-Lite. Table 2 re-prices the same tokens on each provider's "big" and "small" tier; that tier mapping is our assumption. The 16-model market table uses prices re-checked on 2026-11-12 (sources per row in the CSV) and the third-party Artificial Analysis Intelligence Index as a coarse quality signal.

Quality guard. Exact label match for classification; an LLM judge (gemini-4-argon-high, written rubric: correct, complete, policy-safe, full KB in context) for everything else. A lever only counts if the final pipeline still passes the eval; every miss is listed in the source data.

Model sweep. Separately, we ran 20 support prompts (and four other task types, 200 prompts per model) on 7 Gemini models on 11 Nov 2026, judged by gemini-4-carbon-high; a second judge re-scored 20 responses with 98.4% criterion agreement.

2. Results

2.1 Six levers, one bill

Figure 1. Sequential savings on the modelled monthly bill, each call at the list price of the model that served it: Claude Sonnet 5.5 or Gemini 3.1 Flash-Lite. Each bar is the saving of that lever applied after the ones above it; the six bars sum exactly to the difference, $23,117 − $531. Dedupe (−$26) and batch (−$23) read small because earlier levers already removed most of the cost.

Table 1. The waterfall in numbers, as run; the same figures as the live Cost Cutter demo.

Table 2. Re-pricing: the same measured tokens priced on each provider's list, for comparison only; the headline is the as-run row. The big-tier models share a $2/$10 list price per million input/output tokens, so the baseline and the first two levers are identical on every list; the lists differ from routing on, where the small tiers' prices differ.

2.2 Quality held: 78/81 → 76/81

Table 3. Eval by task. Overall 78/81 → 76/81. Two of the 5 final misses are also baseline misses; the other three are completeness or fidelity misses the baseline passed. Cost per resolved support question: $0.0158 → $0.00086.

2.3 The same support prompt on seven models

Figure 2. Measured cost per 1,000 support answers vs rubric quality for six of the seven models, 20 prompts per model, list prices. The seventh, Gemini 3.8 Flash (tiered), is a variant of 3.8 Flash and is in Table 4 only. Higher-reasoning models are not on a different price list so much as they write more tokens.

Table 4. Support task, 7 measured models. "Resolved" divides cost by the pass rate. With 20 items per model, differences of a few points are within noise.

2.4 Sensitivity: caching and routing

Figure 3. Final monthly bill as a share of the $23,117 baseline, by prompt-cache hit rate, with all six levers (white) and without model routing (grey), at as-run prices. Routing matters more than the cache hit rate once prompts are trimmed.

Table 5. “Caching alone” is the monthly saving from caching on the unoptimised baseline. At a 0% hit rate it costs money because of the 1.25× cache-write price of the big model; OpenAI and Google pages list no write premium, so there it is ~$0.

2.5 The market, priced on one support chat

Table 6. 100,000 support chats a month at 650 input and 63 output tokens each (our measured support-prompt size on a non-reasoning model), list prices checked 2026-11-12. The AA index is a third-party general score, not our eval; the cheapest row is not a like-for-like swap. Reasoning models will write more output tokens than 63, so their real cost is higher than this row shows (see Table 4).

3. Scope

One fictional workload, one run per stage at temperature 0, one LLM judge per study; monthly volume is assumed, per-request tokens are measured. Caching assumes a 95% hit rate and ignores cache TTL and storage fees. Prices are list prices on the stated dates and change (Google lists a Gemini 4 Carbon price rise on 2026-12-01). The baseline is a deliberately unoptimised but common shape (big reasoning model on every call, long prompt, 10 retrieved chunks, uncapped output), so a lean stack will see smaller percentages; the order of the levers is the transferable result.

4. Recommendations

• Measure cost per resolved request, from your own usage logs, before changing models.
• Cut tokens first: trim the system prompt (keep every rule), rerank to 3 chunks, cap output and set a low reasoning budget where the eval allows.
• Route classification, summaries and routine questions to a small model, with an escalation rule back to the big one; gate every change on a fixed eval set.
• Turn on prompt caching only for long stable prefixes with a high real hit rate; check the provider's cache-write price.
• Batch anything not user-facing (nightly digests, back-office summaries).

Appendix

Data. r3_waterfall.csv (as-run prices and 3 provider price lists × 6 levers), r3_measured_models.csv (7 models × 10 tasks), r3_market_prices.csv (16 models, source URL per row).

Code. The Cost Cutter engine (30 tests), the LLM Bill Benchmark runner and the sensitivity re-pricing script are not published; we share them on request. Live demos: Cost Cutter, LLM Bill Benchmark.

Cite this report