Third episode of Spending Less on AI. Episode 1, Headroom: the editor between you and the model, set up a tool that promises a lighter bill by sending less text to the model. Episode 2, A tool to spend less on AI made my bill go up, reported the first receipts. This one is not about the tool any more. It is about the number every tool in this market shows you, and why that number no longer tells you what you will pay.
Every product that promises to cut your AI bill shows you the same chart: a line called tokens going down. Sixty percent fewer. Ninety-five percent fewer. The chart is usually true.
The bill is a different document. We opened ours line by line, over 44 working days and four projects, and the two documents disagree. On the project where we sent less text to the model, the text it re-read fell by 46% per round of work and the cost per round went up 14%.
This episode is the anatomy of that disagreement. If you pay for AI by the month, it changes which number you should be asking your team for.
A token is not a price
A token is roughly three-quarters of a word. It is the unit the provider counts. It is also, since 2024, not the unit the provider charges, and this is the whole story.
A coding agent works in rounds. In each round it sends the model the entire conversation so far (every instruction, every file it read, every result it got back) and receives one answer. The conversation grows every round, so by the afternoon each round carries hundreds of thousands of tokens.
Sending all of that at full price would be ruinous, so the provider offers a deal called a cache. Text the model has already seen can be re-read at a tenth of the price of new text. In exchange, the first time a stretch of text is stored, you pay a premium to write it: 1.25 times the new-text price for a five-minute cache, twice the price for an hour-long one. And the answer the model writes costs five times what new text costs.
So one round of work is four lines on the invoice, at four different prices:
| What is billed | Price, relative to new text |
|---|---|
| text re-read from cache | 0.1× |
| new text, never seen | 1× |
| text written to cache (5 min / 1 hour) | 1.25× / 2× |
| the answer the model generates | 5× |
A token on the first line is worth one-fiftieth of a token on the last line. “Tokens went down” is a sentence about the sum of four quantities that should never have been added together.
Where the tokens are, and where the money is
Here is our corpus: 44 days, four projects, 11.9 billion tokens, US$ 10,206 at list price. The first bar is where the tokens are. The second is where the money is.
Ninety-six percent of every token we sent was a re-read from cache. That is the mass a compression tool sees, and the mass it compresses. It carries half the money.
The other half sits in two places a token count barely registers. Cache writes are 4% of the tokens and 42% of the money. The model’s own answers are 0.3% of the tokens and 8% of the money.
This is the first death. A tool that trims the big bar can only ever touch half the bill, and it has to trim a lot of cheap tokens to move it.
Four days, same tokens, three prices
The second death is simpler to see. Take four ordinary working days from one project, on the same model, chosen because each round of work sent almost exactly the same amount of text to the model. If tokens were the price, the four days would cost the same.
Day 1 and day 4 sent the same text and day 4 cost three times as much. The difference is entirely in one line of the invoice: on day 1, three thousand tokens per round were written to cache; on day 4, seventy-two thousand. Everything else, the re-reads and the answers, is nearly identical.
Across all thirty days of this project on this model family, knowing the tokens a day sent explains about half of what it cost. Knowing how much it wrote to cache explains nearly all of it.
Nothing here is a verdict on any tool. It is a description of the invoice. The bill follows the writes. Volume is a bystander.
What the tool does to those four lines
Now the tool. A proxy sits between the agent and the model and shortens what the agent sends: it trims a long log, folds a repeated listing, replaces a verbose search result with a summary that can be expanded on request. It does this well. On our traffic it removed text exactly as advertised.
The question is what that does to the invoice. Below is one round of work, line by line, on the same project at the same price list: the grey bar is a round without the proxy, the blue bar is a round with it.
Read it top to bottom.
First line, re-reads, priced at 0.1×. This is what the proxy compresses, and it worked: nearly half the volume gone. That is the chart the product shows you.
Second line, writes, priced at up to 2×. This is the price of compressing. The provider’s cache is a promise that this exact text will be seen again. The moment the proxy changes the text, the provider sees new text, stores it again, and charges the write premium. Writes nearly tripled.
Third line, the answer, priced at 5×. Longer answers. Part of that is the newer model, not the proxy; it is here because it is on the invoice.
Last line, the total. A token saved on the first line is worth one-twentieth of a token added on the second. The round costs 14% more.
Nothing in this is a defect in anyone’s code. A tool whose job is to change what the model sees is, by construction, a tool that forces re-writes. It saves on the 0.1× line and spends on the 2× line.
Two days on the proxied side is a short run, and the two sides ran on sibling models, which is why the chart says “same price list” rather than “same model”. We will keep measuring. But the direction has held on every cut we have made, and it is the same direction an independent study found in July: across paid, controlled runs of the same coding agent, the configuration that cut delivered tokens by 38% raised the bill by 6.8%, and the correlation between tokens cut and money saved was 0.15. Their estimate of the most that compressing visible text can ever save, given how the cache dominates the bill: about 5%.
What the dashboard calls savings
If the bill goes up, why does every user of these tools report savings? Because the tool’s own dashboard is the receipt most people read, and the dashboard counts differently.
Our proxy’s dashboard, after 3,200 requests, reported US$ 14 saved by compression and US$ 1,667 saved by caching. The second number is the provider’s own discount: the 0.1× line, which every user gets with or without a proxy. Presented side by side, the discount is the headline and the compression is a rounding error, and a reader who does not know the invoice sees a tool that saved sixteen hundred dollars.
The more honest number is three screens deeper, in the tool’s own counters: it removed 1.79 million tokens by compression and lost 2.84 million to cache re-writes it caused. The counter is labelled net tokens: negative. The tool knows. It just does not lead with it.
We are not the first to notice. A developer who ran the same tool for several days found US$ 47 of his US$ 57 “saved” was the provider’s discount, and that only 2% of his traffic was ever compressed. His advice was to read the breakdown instead of the front page. The study above makes the same point in its title.
A correction, in the same spirit
Measuring this cost us a lesson we owe the reader. Until this week, every total in this series was about twice too high. The agent’s log writes one line per piece of an answer, and each line repeats the cost of the whole answer; we were counting lines. We found it while building a screen that lets you open a bill line by line, where identical answers two seconds apart looked wrong. We confirmed it against an independent request log and recounted everything.
The totals in episodes 1 and 2 halve. The conclusions survive, because they were already stated per round and both sides of every comparison were inflated by the same factor. We say it here because the point of this episode is that counting tokens is treacherous, and we are not exempt.
What to ask for instead
For a manager, an executive or a small business owner paying for this, the practical outcome is a change of question.
Do not ask how many tokens were saved. It is the quantity the tools optimise, and it is the quantity the invoice cares least about.
Ask for three things:
- Cost per round of work, by model, per day. It is the only number that survives changes in the amount of work, and the only one on which a before-and-after means anything. Ours is on a screen that opens from one total down to every request, with the arithmetic shown on each line.
- The share of the bill that is cache writes. It is the line that moved our bill, and the line no token chart shows. If it rises, something is changing the text between rounds: a tool, a long pause, a prompt that embeds the time of day.
- Which model answered which kind of task. Across our days, the choice of model moved the bill more than any proxy did, in both directions. It is a lever you own and it costs nothing to pull.
The tool from episodes 1 and 2 stays installed for now, on one project, under a clean test, because two days is not a verdict and we promised to publish the number whichever way it goes. But the series has changed subject. The token is the wrong unit, and the rest of these episodes will be written in the right one.
Method, in brief. Four projects, 44 working days, every model answer read from the agent’s own logs and priced at the provider’s published list rates, including the separate rates for five-minute and one-hour cache writes. Answers deduplicated by response id. The proxied project is compared only against itself, per round, on one model family at one price list. Days in which the proxy ran a configuration later found to be wrong are excluded from the before-and-after and shown nowhere as evidence. All figures are list-price equivalents: we pay a flat monthly plan, and this series is the study we are running before moving to metered use.
Sources: S. Weinberger and A. Hozez, “Token Reduction Is Not Cost Reduction: An Empirical Study of End-to-End Efficiency in API-Based Coding Agents”, arXiv:2607.12161, July 2026. The field report with the US$ 47 of US$ 57 is at russ.cloud, June 2026.