Fourth episode of Spending Less on AI. Episodes 1 and 2, Headroom: the editor between you and the model and A tool to spend less on AI made my bill go up, installed a tool that promised a lighter bill by sending fewer tokens to the model. Episode 3, The Death of Tokens, opened the invoice: the tokens fell 46% and the bill rose 14%, because tokens are not what the provider charges for. This is what we did next.

For three episodes this series tried to spend less on AI by sending less text to the model. It is the obvious idea, every tool in this market sells it, and in our hands it failed. The tool cut what the agent sends by nearly half, and a round of work came out 14% dearer. Episode 3 explained why: a token is what the provider counts, the invoice has four lines at four prices, and the line the tool shrank is the cheapest of the four.

The same episode ended on the one lever that had moved our bill more than any tool, in both directions: which model answered which kind of task. So we stopped saving tokens and pulled that lever instead. This episode is the receipt.

The short answer is yes, it saved money. The long answer is more interesting, because the bill did not fall the way the arithmetic said it would, and the reason points straight at the next episode.

Where the idea came from

It came from a spreadsheet I kept for five weeks, not to study anything, just to know. An AI coding agent had been doing the delivery work on a client platform for months, and I had never looked at the invoice task by task. Thirty-six tasks went in. At list price they came to $5,908.

05001 0001 5002 0002 500ACBDmedian · 24236 tasks, five weeks — one bar per tasksorted by sizerounds of work (each one re-sends the whole conversation)
Thirty-six tasks, one bar each. The smallest cost nine dollars, the largest 945, and between them sits every kind of work a delivery team does in five weeks. The four the article follows are in navy. All thirty-six ran on the same model.

Sorted, the sheet showed one thing before it showed anything else: every task had run on the same model, at the same effort, from the one that took a single round of work to the one that took two and a half thousand. Not because anyone had weighed it. Because that was the setting the session opened with, months earlier, and nobody had touched it since.

Then I went looking for the worst case and found a task that cost $660 to work through fourteen items from an automated report, one at a time. The agent did good work: it traced each item, wrote a short rule to tell a real finding from a false one, and applied it. Twelve of the fourteen were false, and eleven of the twelve were the same false finding. The task needed one judgment and thirteen repetitions of it, and it got the most expensive model doing all fourteen in one long conversation, re-reading everything it had learned about the first thirteen before touching the last. Five times I had to pull it back on course.

what happened — one model, one conversationfive times I had to pull it back on course; two tasks tangled in one thread1 997what the work actually wasOpuswrites the rule, oncew1w2w3w4w5w6w7w8w9w10w11w12w13w1414 × Haiku · in parallel, one item eacha clean context every time, so nothing can get tangledOpusreviews the two real ones2 come backthe other 12 were the same mistake
The repeated task, drawn twice. Above, the $660 as it was actually spent, with a flag wherever I had to pull the conversation back on course. Below, the shape the work had all along: write the rule once, apply it fourteen times, judge twice.

And then the opposite case, which is why the answer is not “use the cheap model”. A $304 task opened with one sentence: something the system sends arrives wrong on the other side. The agent doubted the sentence, read years of history, and found that a piece everybody assumed was added automatically had stopped being added during an old upgrade. Nine things were broken, not the one in the request. No cheap model asks that question. Three hundred dollars was the price of doubt, and the best three hundred of the five weeks.

Two kinds of work, then. Judgment, which wants the expensive model. And repetition, which wants anything but. One price for both.

What a round of work costs

To see why the price matters so much, you have to see what a round is.

one round of workthe conversationtokens re-sent, average509 000the modelreads all of it, againthe answertokens written, average1 000where the 37 cents gore-reading what it already saw · 73%caching · 20%the answer · 7%Thinking lives inside that 7%. What an agent costs is not what it writes — it is what it re-reads.which leaves exactly two leversThe model sets the price of every line: at the time, Opus $5, Sonnet $2, Haiku $1 per million tokens read.The effort level sets how many rounds there are: lower effort, fewer and larger steps, fewer rounds.Average round across the five weeks: 509,000 tokens in, 1,000 out, 37 cents.
One round of work, taken apart. The agent re-sends half a million tokens in order to write a thousand — so three quarters of the bill is re-reading, and the model you choose sets the price of every re-read.

An agent has no memory between one action and the next. Every time it does something, it sends the whole conversation back to the model and asks what to do now. That is one round, and by the afternoon each one carries the equivalent of a 350-page book. The book is four lines on the invoice: re-read from the cache at a tenth of the price of new text, new text at full price, text written into the cache at up to twice, and the answer at five times. In the five weeks, with nothing disturbing the cache, re-reading was 73 cents of every dollar and the answer, thinking included, seven.

The model multiplies all four lines at once. At the time, Opus read at five dollars per million tokens, Sonnet at two, Haiku at one. Same book, same number of rounds, less than half the cost on Sonnet and a fifth on Haiku. The effort level sets how many rounds there are: lower effort, fewer and larger steps, fewer books re-sent.

That is the whole idea. Keep the expensive model for the judgment, hand the repetition to a cheaper one, and let somebody decide which is which, task by task.

What we changed, and for how long

At the end of September, on the same client platform, one model for everything was replaced by a lead and five helpers.

the leadOpus · the session itselfreads the request, splits the work, holds the scope, judges what comes backreadermodel: sonneteffort: lowmaxTurns: 25reads, never writesbuildermodel: sonneteffort: mediummaxTurns: 80one settled changetestermodel: sonneteffort: highmaxTurns: 120level set at hand-offreviewermodel: inheriteffort: highmaxTurns: 40the lead's own levelQA leadmodel: inheriteffort: highmaxTurns: 160dispatches the testersand the lead can override the model on any single hand-off“This batch is trivial — run it on Sonnet.”The file sets the default. The hand-off sets the exception. The lead holds both.These are the real settings of the five helper files, not a metaphor.
What replaced one model for everything, as it was actually configured: a lead that decides and never executes, three helpers on the cheaper model that read, build and test, and two adversaries — the reviewer and the QA lead — that deliberately run at the lead's own level.

The lead is the session itself, on the expensive model, and it does three things only: reads the request and decides whether it can be trusted, splits the work and picks who does each piece, and holds the scope while the others work, judging what comes back. It does not read fourteen items.

The helpers are five short configuration files, each naming a model, an effort level and a turn limit. A reader on Sonnet at low effort, which reads and never writes. A builder on Sonnet at medium effort, which implements one settled change and never negotiates with the request. A tester on Sonnet, dispatched by a QA lead that decides what the testing has to cover. And a reviewer that attacks the plan before any work is handed out. The reviewer and the QA lead run at the lead’s own level, on purpose: a reviewer below the lead can only review spelling.

And because a big task changes shape while it runs, the lead can override the model on any single hand-off. When a batch turns out to be trivial it says Haiku, this one and moves on. The file sets the default; the hand-off sets the exception.

This has been running since 29 September. The invoice below compares the twenty-four working days before it, all on the expensive model, with the sixteen days since, ten of them working days, on the same platform with the same operator.

What the invoice says

what one round cost, before and afterUS$ per round · list pricebefore · the session alone, on Opus0.52after · by rolethe lead0.24QA lead · lead's level0.15tester, while it inherited the lead's model0.13reviewer · lead's level0.06tester · Sonnet0.047builder · Sonnet0.04reader · Sonnet0.035builder · Haiku, by hand-off0.01a working day: US$ 163 before, US$ 137 after — with three times more rounds donerounds on the most expensive model: nine in ten before, about one in three after
The invoice, per round, before and after the lead and the helpers went in. Before, every round was the session itself on the expensive model. After, the lead's rounds cost half as much and the rounds that moved to the helpers cost a tenth. The red bar is the tester during the weeks it silently inherited the lead's model.

Before, every round was the session itself on the expensive model, and a round cost 52 cents. After, the lead’s own rounds cost 24 — part of that is a newer generation of the same model, priced a fifth lower, and part is a lead that takes fewer, larger steps because it no longer executes. The rounds that moved to the helpers cost four to five cents. The batches the lead sent to Haiku by hand cost one. Nine in ten rounds ran on the most expensive model before; about one in three after.

The invoice also showed me something I had not configured. The tester had no model of its own: it inherited whatever the session was running, which was the most expensive one, and it did 1,441 rounds that way at 13 cents each before anyone noticed. Since the file got the one line it was missing, the same job costs under five cents a round. Nobody made a bad call there. Nobody made a call, and the default is invisible until you go looking.

So: did it save money?

Yes. Per round of work, a quarter of the price: 37 cents on average before, 9 after. Per working day, $163 before and $137 after, 16% less — while the agent did three times as many rounds, because helpers take many small steps where the session took few large ones. Read it as: the day cost less and more got done, not as three times the work.

Sixteen per cent is less than the thirty the arithmetic promised when this article was still a draft, and the invoice says why. The lead’s rounds cost 24 cents and it still does about four hundred of them a day. On a quiet day it is seventy per cent of the bill. We moved from everything on the expensive model to the lead on the expensive model, and the lead is now the bill. That is the next lever, and the next episode.

Put the two experiments side by side. The tool of episodes 1 to 3 cut tokens by 46% and made a round 14% dearer. Managing models did not touch the tokens — the agent sends more than ever — and made a round 75% cheaper. Spending less on AI was never about tokens. It is about who does the work, and what you pay them per round.

The spreadsheet is still open.

Method, in short. One client platform, the same operator throughout. The five weeks are 36 tasks with every round attributed to the task that asked for it, 15 August to 21 September. The before and after is every round of the same platform from 31 August to 28 September against every round from 29 September to 14 October, each priced at the provider’s published rate for the model that answered it and attributed to the role that ran it, from the agent’s own logs. Part of the after window still ran through the proxy of episodes 1 to 3, which makes a round dearer, so the after is, if anything, overstated; the days after the proxy was removed show the same per-round levels. All values are list-price equivalents: we pay a fixed monthly plan, and this series is the study we do before moving to metered use.