Everyone talks about what an agent can do. Almost no one talks about what it costs to let it try. This is the other half of the agentic AI story: the meter that runs underneath every loop, every tool call, every retry, and every extra turn a model takes while it figures out what you actually meant. Autonomy is not free, and the bill does not arrive the way most engineering teams expect.
A single chat completion has a cost you can estimate on a napkin: count the input tokens, count the output tokens, multiply by the per-token price, done. An agent breaks that napkin math completely. It reads, it plans, it calls a tool, it reads the tool's answer, it reasons about what to do next, and it repeats that cycle an unpredictable number of times before it decides the task is finished. Each of those steps is its own model call, and every one of those calls re-sends a context window that has been quietly growing since the loop began. The cost of an agent is not the cost of an answer. It is the cost of a conversation the system is having with itself, on your account, for as long as it takes to converge.
This piece is a tour of that economics: where the money and the milliseconds actually go inside an agent loop, why the obvious optimizations are not always the effective ones, and what a team building production agents should actually measure and control. None of this requires exotic infrastructure. It requires taking the token bill as seriously as you take the correctness of the agent's reasoning, because past a certain scale, the two problems become the same problem.
Why agents cost differently than chat
A conventional LLM feature costs roughly what you'd expect from its inputs and outputs, because it makes one call and stops. An agent is defined by the opposite property: it keeps calling the model until some internal condition says the task is done, and neither you nor the agent knows in advance how many calls that will take. A question that would cost a few hundred tokens as a single completion can cost tens of thousands of tokens once it becomes the seed of a loop that reads documents, calls three tools, hits an error, retries, and reasons over the combined output of everything it has seen so far.
The unpredictability is the core economic fact of agentic systems, and it changes how you have to think about pricing a feature. You cannot quote a customer, a product manager, or your own finance team a fixed cost per request, because the cost is a distribution, not a number. Simple tasks resolve in one or two steps and cost next to nothing. Hard tasks, ambiguous tasks, and tasks that hit a flaky external API can balloon into dozens of steps, and the tail of that distribution is where most of an agent product's compute budget actually goes. Teams that only look at the median cost per request are almost always underestimating what the feature costs to run, because the mean is dragged hard by a long right tail of expensive outliers.
This is also why agent cost cannot be reasoned about the way infrastructure engineers reason about compute. A slow database query costs you latency but a roughly fixed amount of CPU. A slow agent loop costs you latency and a compounding amount of tokens, because every extra step re-reads everything that came before it. The two failure modes stack: the loop gets slower and each step in the slower loop is itself more expensive than the step before it. Understanding that compounding is the single most important mental model in this entire subject, and the rest of this piece is largely about how to manage it.
The cost of an agent is not the cost of an answer. It is the cost of a conversation the system is having with itself, for as long as it takes to converge.
The anatomy of an agent's token bill
To manage a cost you first have to see where it comes from, and an agent's token bill has more line items than most engineers assume before they've actually profiled one. Start with the obvious ones: the system prompt, which is resent on every single call in the loop, and the user's original request, which likewise persists across every turn. Neither of these grows during the loop, but neither is free, and a verbose system prompt with extensive instructions and examples is a tax you pay on every step, not once per session.
System prompts and tool schemas
The less obvious cost sits in the tool definitions. Every tool available to an agent has to be described to the model in every call, typically as a structured schema with a name, a description, and a parameter list, so the model knows what it can do and how to call it. An agent with fifteen tools, each with a paragraph of description and a handful of typed parameters, can easily spend one or two thousand tokens on tool schemas alone before a single word of actual conversation has occurred. That cost is invisible in a demo with three tools and brutal in a production system with thirty, and it is paid again on every step of every loop, whether or not that particular step ends up using any of those tools at all.
The fix is not to stop giving the agent tools; it is to stop giving it more tools than the current step could plausibly need. Many production systems now scope the visible toolset dynamically, showing the agent only the tools relevant to its current phase of work rather than the entire catalog on every call. This is the same instinct that drives context engineering in retrieval systems, applied to actions instead of documents: the model doesn't need everything it might ever need, it needs what is relevant right now, and everything else is a token tax with no offsetting benefit.
The context accumulation problem
The largest and least intuitive cost driver is the growth of the context itself as the loop progresses. Each tool call returns a result, and that result gets appended to the conversation so the model can reason about it on the next step. A web search might return several thousand tokens of snippets. A database query might return a table. A file read might return an entire document. None of that gets discarded by default; it accumulates, turn after turn, and every subsequent call in the loop re-sends the entire accumulated history, including everything from every prior step, because the model has no memory between calls beyond what you hand it back in the prompt.
This means the tenth step of a ten-step agent loop is not merely one call; it is a call carrying the full weight of the nine calls before it. If each tool result averages two thousand tokens, step ten is re-transmitting eighteen thousand tokens of prior tool output just to ask one more question, on top of the system prompt and tool schemas that were already there. Multiply that by the price per input token and the picture becomes clear: the cost of an agent loop does not grow linearly with the number of steps, it grows roughly quadratically, because each additional step both adds new tokens and re-pays for every token that came before it. A loop that feels like it is only taking a few extra steps to be thorough can be costing multiples of what a tighter loop would cost, and the relationship is nonlinear enough that shaving two steps off a ten-step loop can matter more than it looks like it should.
The multiplier: how loops compound cost
It's worth making the quadratic-growth point concrete, because intuition about linear costs is exactly the intuition that misleads people here. Picture an agent loop where each step produces roughly the same amount of new content, call it a constant number of tokens per step, and where every step resends everything generated so far. The total tokens processed across an n-step loop is not proportional to n; it is proportional to the sum of the first n integers, which grows proportional to n squared. Doubling the number of steps in a loop does not double the cost. It roughly quadruples it.
This is the mathematical reason agent cost overruns tend to feel sudden rather than gradual. A loop humming along at four or five steps looks perfectly affordable, and then a harder input pushes it to twelve or fifteen steps, and the bill for that one request is not three times higher, it is closer to an order of magnitude higher. Teams that provision budget based on average step count without modeling the tail get blindsided by exactly this effect the first time a genuinely hard task hits production traffic. Anyone setting a budget for an agentic feature should be sizing for the shape of that tail, not the shape of the median, because the tail is where the money actually goes.
The practical response to compounding cost is not merely to make the loop shorter, though that helps; it is to break the assumption that every step must carry the full history of every prior step. Summarizing or discarding stale tool output, keeping only the parts of a prior result that are still relevant to the current decision, and closing out sub-tasks so their intermediate scaffolding doesn't ride along for the rest of the loop are all ways of flattening the curve back toward linear. This is context engineering again, wearing a different hat: the discipline of deciding what belongs in the window applies exactly as much to an agent's own history as it does to retrieved documents.
Prompt caching and its economics
The single highest-leverage lever most teams underuse is prompt caching. Because an agent loop resends a large, mostly-unchanged prefix on every step, that is close to the ideal workload for caching: a stable system prompt, a stable set of tool definitions, and a growing-but-append-only conversation history. When the provider can cache the processed representation of that stable prefix and charge a fraction of the normal input price for cache hits, the quadratic cost curve described above gets a much smaller constant in front of it, even though the token count itself hasn't changed.
The catch is that caching rewards a particular discipline in how you construct the prompt. Anything you place before the point where the content starts changing has to actually stay identical, byte for byte, across calls, or the cache misses and you pay full price with none of the benefit. This argues for a specific ordering: system instructions and tool schemas first, since they truly don't change within a session; then the stable parts of the conversation; and only at the very end the parts that are genuinely new on this step. Teams that interleave dynamic content, such as a live timestamp or a per-call random identifier, into the early part of the prompt without realizing it are silently defeating their own cache on every single call, and it is a surprisingly common and surprisingly expensive mistake to audit for.
Caching also interacts with tool scoping in a way that's worth flagging. If you dynamically vary which tools are visible to the model from step to step, in order to save the tokens described earlier, you are also changing the prefix that would otherwise have been cacheable, because the tool schema block is part of that stable prefix. There is a genuine tension between showing only the tools this step needs and keeping the prefix identical so the cache hits, and the right answer depends on the relative price of a cache miss versus the price of the unused tool tokens you'd otherwise be carrying. Neither optimization is free, and treating them as independent, additive wins rather than a tradeoff to be measured is a common way teams leave money on the table.
Model routing as a cost lever
Model routing is usually framed as a quality-and-capability decision: use the frontier model for the step that needs deep reasoning and a smaller, cheaper model for the step that doesn't. It is worth stating plainly that this is also, first and foremost, a cost decision, and often the single largest cost lever available in an agent's architecture, because the price gap between a frontier model and a small fast model is typically far larger than any prompt-level optimization could achieve on its own.
The steps inside a typical agent loop are not uniformly difficult. Deciding which of fifteen tools to call given a clear instruction is a much easier problem than synthesizing a final answer from ambiguous, partially contradictory evidence. Classifying whether a tool result actually answered the question is easier still. Routing the easy, high-frequency steps to a small model and reserving the expensive model for the steps that carry real reasoning load can cut the blended cost of a loop dramatically without touching the quality of the parts that matter, because most agent loops spend most of their steps on exactly the kind of routine, low-ambiguity decisions a smaller model handles perfectly well.
The engineering cost of doing this properly is nontrivial, because it requires classifying steps by difficulty in a way that's itself cheap and reliable, and getting that classification wrong in the direction of under-provisioning quality will surface as agent failures rather than as a line item on a bill, which makes the failure mode harder to notice and more expensive to debug after the fact. The practical starting point most teams find workable is coarse rather than fine: route by step type rather than by a learned per-step difficulty score, since a fixed mapping where tool selection uses the small model and final synthesis uses the large model captures most of the available savings with a fraction of the complexity of a fully adaptive router, and it's far easier to reason about and audit.
Latency is a cost too
It's tempting to treat latency and dollar cost as separate concerns, one an engineering metric and the other a finance metric, but in an agentic system they are close to the same thing measured in different units, because both scale with the same underlying quantity: the number of sequential model calls a task requires. A loop that takes twelve steps is both expensive and slow for the same structural reason, and optimizing one without thinking about the other usually means you've only half-solved the problem.
The user-facing cost of latency is easy to underweight because it doesn't show up on an invoice. A task that could technically complete in eight seconds but instead takes forty because the agent took nine sequential steps to get there is not free just because the token cost was survivable; it is costing you in abandonment, in perceived unreliability, and in the user's willingness to trust the system with the next task. Products that expose agent latency directly to end users, without streaming intermediate progress or setting expectations about what thinking looks like, tend to see engagement drop off well before the dollar cost of the feature becomes a problem, which means latency often becomes the binding constraint on agent design before cost does, even in systems where cost was the thing everyone was watching.
Reducing latency and reducing cost frequently point at the same fixes, which is good news for anyone trying to prioritize: shorter loops, smaller contexts, cached prefixes, and cheaper models for routine steps all cut both dimensions simultaneously. Where they diverge is parallelism, covered next, which can reduce wall-clock latency while increasing total token spend, and knowing which of the two constraints binds for your product determines which side of that tradeoff you should take.
Tool calls: hidden costs beyond tokens
Every tool call an agent makes carries costs beyond the tokens spent describing and invoking it. If the tool itself calls an external API, that API has its own price, and an agent that calls a paid search or data API four times in the course of answering one question is paying for four external calls on top of whatever the model provider charges, a cost that's easy to lose track of because it's billed on a completely separate invoice from the model usage.
There is also a latency cost specific to tools that has nothing to do with the model at all. A tool call means the agent loop is now waiting on a network round trip, a database query, or a third-party service's own processing time, and that wait is pure dead time from the model's perspective, contributing nothing to reasoning quality while still extending the wall-clock duration of the loop and, if the provider bills for elapsed session time in any form, extending the cost as well. A slow tool inside a fast agent turns the agent slow, and no amount of prompt optimization fixes a tool that takes six seconds to respond.
The design implication is that tool selection deserves the same cost scrutiny as model selection. Preferring a cheap, fast, slightly-less-comprehensive data source over an expensive, slow, exhaustive one is frequently the right tradeoff for an agent that will call that tool many times across many loops, even if a human doing the equivalent research once would reasonably choose the expensive option. The economics of a tool used by an autonomous loop thousands of times a day are not the economics of a tool used by a person a few times a week, and importing human intuition about which data source is better without adjusting for call volume is a common source of surprisingly large infrastructure bills downstream of what looked like a model-cost decision.
Retries and the cost of imperfection
No agent loop is perfectly reliable, and the handling of failure is itself a cost center that's easy to design carelessly. A tool call that times out, a malformed function-call argument that fails validation, a rate limit hit on an external API: all of these trigger some form of retry, and a naive retry strategy resends the full accumulated context every time it tries again, meaning a single failure at step eight of a loop doesn't just cost the price of one retried call, it costs the price of retransmitting everything that led up to step eight, again.
This compounds in the same quadratic way the base loop does, except now it's compounding on top of already-compounded cost, which is why unbounded retries are one of the more common causes of a runaway agent bill in practice. A poorly configured retry policy, especially one without exponential backoff or a hard cap on attempts, can turn one transient failure into a cascade that reprocesses the same expensive context five or six times before giving up, and if the underlying cause is systemic rather than transient, such as a genuinely broken tool or an exhausted rate limit, that cascade can repeat across every concurrent request hitting the same failure at once.
The mitigations are not exotic, but they have to be deliberate rather than assumed. Cap retry attempts explicitly rather than relying on a framework's default, which is often more generous than it should be for a loop this expensive to resend. Distinguish retryable failures, like a timeout, from non-retryable ones, like a validation error the model itself needs to fix by changing its approach, because retrying the second kind of failure without changing the input just reproduces the same failure at full price. And treat a spike in retry rate as a cost alarm as much as a reliability alarm, because by the time it shows up as a reliability problem in your dashboards, it has usually already shown up as a cost problem in your bill.
Parallel vs sequential: a latency-cost tradeoff
When an agent needs several independent pieces of information, such as the current weather, a stock price, and a calendar lookup, it can either fetch them one at a time in sequence or issue all three tool calls at once and wait for the results together. Parallel tool calling is a genuine and often dramatic latency win, because the wall-clock time collapses to the slowest of the three calls rather than the sum of all three, and this alone is why most modern agent frameworks default to encouraging it wherever the calls are truly independent.
It is not, however, automatically a cost win, and treating it as one is a common category error. Firing three tool calls in parallel means paying for three tool calls regardless of whether the agent turns out to need all three answers to complete the task; a sequential approach at least has the option of stopping early once the first result makes the others unnecessary. For a task where the calls are genuinely all required, parallelism is close to free from a token perspective and a clear win on latency. For a task where some of those calls are exploratory and might turn out to be unneeded once earlier results come in, parallelism trades a small amount of wasted spend for a real latency improvement, and whether that trade is worth it depends entirely on how expensive the individual calls are and how often the exploratory ones actually turn out to be unnecessary.
The decision, in other words, is not that parallel is always faster so always parallelize, it's a genuine tradeoff to evaluate per tool and per task shape: how independent are the calls really, how expensive is each one individually, and how much does the latency improvement matter to the product experience. Defaulting to parallel for calls that are cheap and genuinely independent, while keeping conditionally-needed or expensive calls sequential and gated behind the results that determine whether they're needed at all, captures most of the latency benefit without paying for work that a smarter ordering would have skipped entirely.
A worked example: pricing out an agent loop
Numbers make this concrete in a way abstract description doesn't, so it's worth walking through a stylized example, understanding that actual prices vary by provider and change over time, and the point is the shape of the calculation rather than any specific dollar figure. Imagine a research agent with a two-thousand-token system prompt and tool schema block, answering a moderately complex question that takes six steps to resolve, where each step's tool result averages fifteen hundred tokens and each step's model reasoning and response averages three hundred tokens.
Without caching, step one costs roughly twenty-three hundred input tokens, the system prompt plus the original question. Step two now carries the system prompt, the question, and step one's tool result and response, pushing input tokens to around four thousand. By step six, the loop is resending the system prompt plus five full rounds of accumulated tool results and responses, somewhere north of eleven thousand input tokens for that single step alone. Summed across all six steps, the loop processes on the order of forty thousand input tokens before it produces a final answer, for a task that, described in a sentence, sounds like it should have cost a few thousand.
Now apply prompt caching to the stable prefix, and route the tool-selection reasoning at each step to a smaller model while reserving the larger model for the final synthesis step, and that forty-thousand-token bill can often be cut by more than half without any change to what the agent actually does or how well it does it, purely through infrastructure-level decisions about how the same information gets priced. That gap, between the naive implementation and the cost-engineered one, is the entire subject of this piece compressed into one example, and it's a large enough gap that it should change how any team prioritizes engineering effort on an agentic feature that's headed toward real production volume.
Observability for cost
You cannot manage what you cannot see, and cost in an agentic system is unusually easy to lose track of because it's spread across many small calls rather than concentrated in one obvious place. The foundational practice is tracing every step of every agent loop with enough granularity to attribute tokens, dollars, and latency to the specific step, tool, and model that produced them, not just the aggregate cost of the whole request. Without that granularity, a cost problem shows up as this feature got more expensive last week with no way to say which of a dozen possible causes is responsible.
Per-step attribution turns a vague sense that something got expensive into a specific, actionable finding: this particular tool started returning twice as much text after an upstream API change, this particular loop stopped hitting the prompt cache after an unrelated prompt edit moved a timestamp earlier in the prefix, this particular task category started taking four more steps on average after a new class of user question started arriving. Every one of those is a five-minute fix once you can see it and a multi-day investigation if all you have is an aggregate bill that went up.
Budgets, circuit breakers, and guardrails
Observability tells you what happened. Budgets and circuit breakers stop the worst outcomes from happening in the first place, and any agent given real autonomy in production needs both, because an ungoverned loop has no innate concept of whether a task isn't worth what it's costing to pursue. Left alone, a struggling agent will keep trying, keep calling tools, and keep accumulating context, because from its perspective persistence looks identical to progress right up until the moment the task either succeeds or the process is terminated externally.
A hard step cap is the simplest and most important guardrail: a maximum number of iterations the loop is allowed before it's forced to stop and return whatever it has, regardless of whether it believes the task is complete. This single control bounds the worst case of the quadratic cost curve described earlier and turns an unbounded tail risk into a known maximum, which is the difference between a cost you can budget for and a cost you can only discover after the fact. A token or dollar budget per request serves the same purpose from a different angle, and the two together catch failure modes that either one alone would miss, since a loop can blow a token budget in fewer steps than the step cap allows for, or vice versa.
Beyond hard caps, softer guardrails add real value: a rising-cost alert that flags a request once it's consumed some multiple of the expected cost for its task type, even before it hits the hard ceiling, giving a human or a supervising process the chance to intervene early; a stagnation check that halts a loop making no measurable progress across several consecutive steps, distinct from a loop that's merely taking a long time to make real progress; and a per-user or per-session rate limit that prevents one runaway task, or one malicious actor deliberately trying to trigger expensive loops, from consuming a disproportionate share of the budget. None of these guardrails are exotic engineering, but all of them require being built in deliberately, because the default behavior of an agent loop, absent any of them, is to keep going.
Strategies that actually reduce cost
Pulling the threads of this piece together, the highest-leverage moves for teams building production agents cluster around a small number of ideas, and none of them require exotic infrastructure. Cache the stable prefix and construct prompts deliberately so that caching actually works, since this is close to a free win once the discipline is in place. Scope tools dynamically so the model only carries the schemas it needs for the current phase of work, rather than paying the tax of the full catalog on every step. Route steps to the cheapest model that can reliably handle them, reserving the frontier model for the steps that carry real reasoning load. Summarize or discard stale tool output instead of letting it ride along for the rest of the loop, which is the single biggest lever against the quadratic cost curve. And cap retries explicitly, distinguishing failures worth retrying from failures that need a different approach entirely.
None of these are one-time fixes, either. An agent's cost profile drifts as tools change, as the model provider adjusts pricing, and as usage patterns shift the mix of easy and hard tasks hitting the system. Treating cost optimization as a project that gets finished, rather than a discipline that gets maintained alongside correctness and latency, is how a well-tuned agent quietly becomes an expensive one again eighteen months later, with nobody quite able to point to the day it happened.
Procurement and pricing models
Everything above assumes a per-token pricing model, because that's still how most teams buy model access, but it's worth stepping back to note that per-token pricing is not a law of nature. It's a specific commercial choice that happens to map awkwardly onto agentic workloads, because it charges for exactly the thing an agent loop is worst at controlling: the volume of context re-transmitted on every step. A pricing model built around single-turn chat completions was a reasonable fit for the workloads that existed when it was designed, and it becomes a worse fit every time a team pushes an agent loop further into production.
Alternative commercial arrangements are becoming more common precisely because of this mismatch. Outcome-based pricing, where a vendor charges per resolved task rather than per token, shifts the unpredictability of agent cost from the buyer to the seller, which sounds appealing until you notice that the seller will price in a margin for that unpredictability, and the price for a hard task and an easy task converges toward something closer to the average of the distribution described earlier in this piece. Committed-spend contracts with volume discounts reward teams that can forecast their usage accurately, which is itself a reason to invest in the observability practices described above, since you cannot negotiate a good committed-spend deal without a credible forecast to negotiate from. And multi-provider strategies, where a team routes different workloads to different model providers based on price and capability, add real savings potential at the cost of real engineering complexity in maintaining prompts and evaluations across providers that don't behave identically.
None of these arrangements eliminate the underlying economics; they redistribute who bears the risk of the long tail. A team evaluating a new pricing model should ask the same question a team evaluating its own architecture should ask: does this arrangement reward the behaviors that actually reduce total cost, like shorter loops and better caching, or does it just move the number to a different line on the invoice while leaving the underlying inefficiency untouched. The procurement conversation and the engineering conversation are, in the end, the same conversation, and teams that treat them as separate tend to end up with a contract that's misaligned with the system it's paying for.
Where the cost curve is heading
Three forces are pulling on agent economics at once, and it's worth naming them separately because they pull in different directions and won't resolve at the same pace. The first is falling per-token prices, which have declined sharply as models have matured and competition among providers has intensified; this trend alone makes agent loops that were uneconomical a year ago viable today, and there's little reason to expect it to stop soon. The second is rising context lengths and increasingly ambitious agent tasks, which push in the opposite direction: as agents are trusted with longer-running, more complex work, the loops get longer, the context windows get fuller, and the quadratic cost curve described throughout this piece gets more room to compound before a task resolves.
The third force, and the one most likely to reshape this entire discussion, is architectural rather than commercial: infrastructure purpose-built for agentic workloads rather than adapted from chat-completion infrastructure. Caching systems designed around the specific access patterns of agent loops, rather than generic prefix caching bolted onto a chat API, promise better hit rates for less prompt-construction discipline. Serving infrastructure that treats a multi-step loop as a single billable unit rather than a sequence of independent calls could in principle eliminate much of the re-transmission overhead that drives the quadratic curve in the first place, by keeping relevant state resident between steps instead of re-sending it. None of this is guaranteed to arrive on any particular timeline, and teams building agents today still have to operate under the current constraints, but it's a reasonable bet that the awkward mapping between per-token chat pricing and agentic workloads described in the previous section is a transitional state rather than a permanent one.
What seems durable, regardless of how those specific trends play out, is the underlying shape of the problem: any system that takes multiple sequential actions to accomplish a task will have a cost structure shaped by how many actions it takes and how much context each action carries, and the teams that internalize that structural fact and design against it will keep having an advantage over the teams that treat cost as an afterthought to be optimized once the demo works, no matter how the specific numbers move.
The bottom line
Autonomy sells itself on what an agent can accomplish without being asked twice, and that pitch is real: agents genuinely do things a single chat completion cannot, chaining research, judgment, and action into outcomes no one has to babysit step by step. But every one of those steps has a meter running underneath it, and the teams that build the most capable agents are, almost without exception, the same teams that took the meter seriously from the start rather than treating it as a problem to solve after the product worked. Cost engineering is not a tax on ambition; it's what makes ambitious agent products survive contact with real usage, real scale, and real edge cases long enough to prove their value.
The practical takeaway is not complicated, even if the underlying mechanics are: know where an agent loop's tokens go, cache what's stable, route what's easy to a cheaper model, bound what's unbounded, and watch the tail of the distribution rather than just its median. None of that requires a research breakthrough. It requires treating the economics of autonomy with the same rigor most teams already apply to its correctness, because past a certain point of adoption, the two stop being separable, and a system that thinks brilliantly but costs unpredictably is not a system anyone can afford to trust with the next task.