New on AI SW Factory: token cost fundamentals for coding agents

The short version#

I have published a second module on AI SW Factory: Token cost fundamentals. It is ten short pages on what a coding agent costs to run and why. It is free and vendor-neutral, and it uses token counts rather than prices, so it stays true when the price lists change.

Most advice about agent cost is a list of tricks with no reasons attached: use a smaller model, compact more often, keep prompts short. Some of it helps, some does nothing, and some moves the cost somewhere you are not looking. The module goes the other way. It teaches the few mechanics behind every token bill, and every tip on its pages follows from one of them and says what you give up for it.

• 10concepts, in three groups
• ~10 minper concept
• 1running example team
• 0prices quoted: only token counts and orderings

Four facts explain most of a bill#

When I measured my own sessions with Agent Tokenwatch, and later in the Kubernetes CLI-versus-MCP study, the expensive moments kept coming back to the same four facts:

1. Every call re-reads the whole conversation. The model keeps nothing between calls. A one-line message late in a long session pays to read everything before it, and one reply can contain several calls.
2. Part of that re-read comes from a cache. The provider serves the unchanged start of the input cheaply. The match is exact from the first token, so one early change, such as a toggled tool, makes everything after it fresh again.
3. The cache expires. After a break longer than its lifetime, the first call back rebuilds the whole prefix at full price. Nothing in the chat shows that it happened.
4. Output is generated one token at a time. That makes it the most expensive kind of token, and reasoning tokens usually count as output.

The four kinds of token are priced in an order that holds across providers, even though the exact multiples do not:

Four kinds of token, cheapest to priciest

1. 01Cache readReplays a prefix the provider already holds.
2. 02Fresh inputText read for the first time, in parallel.
3. 03Cache writePays up front to hold a prefix for next time.
4. 04OutputGenerated one token after another.

That ordering explains why two sessions with the same token count can differ in cost by almost four times. The module works through a 200,000-token session twice with the same output, once served mostly from the cache and once mostly as fresh input:

Same 200,000 tokens, different kinds

• Mostly cache reads56,500 units
• Mostly fresh input216,000 units

The path through the module#

The ten concepts come in three groups, and each builds on the one before.

• Foundations: what you actually pay for. Tokens, turns and context, Why tokens cost differently.
• Mechanics: why re-reading is cheap, what breaks it, and why replies get dearer. Prompt caching, Cache lifetime, Context growth.
• Practice: the choices you can change, and a way to check them. Model and reasoning choice, Prompts and exploration, Automation at scale, Measuring cost, Your first cost experiment.

Every concept page has the same shape: why it matters, how it works, a worked example, what helps and what hurts, and the trade-offs. The trade-offs section is the part I care about most. Cheaper can mean slower, lossier or riskier, and a cheap session that produces the wrong change is not a saving.

It ends with an experiment, not a tip#

The last page turns the module into a method. Measure a baseline, change one habit, run it on similar work, and compare cost per finished task together with a quality check such as tests failing after merge. Keep the change only if cost drops and the work stays as good. “Drop” is a valid result.

A first cost experiment

1. 01MeasureCost per finished task, by token kind, over a baseline window.
2. 02Change one thingThe biggest driver first, nothing else at the same time.
3. 03RunSimilar work, enough tasks to see past normal variation.
4. 04DecideCompare cost and quality together, then keep or drop it.

This is the method I used in the Kubernetes study, and the module cites it for two traps that real data shows better than any illustration. One change saved 38% in the first round and cost 45% more in the second, so a single before-and-after comparison would have proved whichever story you wanted. An installed output-compression tool turned out to remove about a tenth of a percent of the output, which is why installing an optimization is not evidence that it worked.

What it is not#

It is not a price list or a vendor comparison. Exact rates come from your provider and change, so the pages teach orderings and rules of thumb instead. It is also not a set of tricks to apply blindly. Every lever on the pages has a cost of its own: a fresh session forgets, a filter can hide the cause, a summary is lossy, and a smaller model can cost more in rework than it saves.

Start here#

• Read it at ai-sw-factory.mellicci.dev/cost. It assumes the agent loop from Module 1, and is worth reading in order.
• To see the same mechanics in your own sessions, Agent Tokenwatch shows tokens and cache health (warm or cold) on the agent’s status line. It records token counts and structure locally, never prompts, code or transcripts:

npm install -g agent-tokenwatch
tokenwatch install --agents all --scope user
tokenwatch doctor

If one page changes one habit of yours, and you measure that it paid off, the module did its job.