What Does an AI Agent Really Cost to Run? A Cost-per-Successful-Task Model

• AI/ML
• Last Updated: September 24, 2026

by Patel Akash

Let AI summarize this article instantly

ChatGPT Perplexity Claude Grok Gemini

Most AI agent budgets are built on one number, the provider’s price per million tokens, and most of them are wrong by an order of magnitude. Token price describes what one model call costs. It says nothing about how many calls an agent makes per task, how much context it resends on each call, how often it fails, or who cleans up when it does.

We built a cost model for three common agent workloads to see what actually drives the bill. The finding was consistent across all three: the per-call price understated the true unit cost by 19 to 72 times, and in two of the three workloads, model tokens were less than a quarter of the total. This article walks through the model, the math, and the levers that matter, so you can budget an agent before it reaches production rather than after the first invoice.

Why price per token is the wrong starting number

A provider price sheet answers a narrow question: what does it cost to send a given number of tokens to a model and receive a given number back? An agent is not one call. A single customer request can trigger a planning call, several tool calls whose results are fed back into the model, a verification pass, and a final response, with the full conversation history attached to each request.

Three properties of agent workloads make per-token thinking misleading.

Calls multiply. Every step in an agent loop is a separate billed request. A six-step task is at least six calls, plus any guardrail or routing calls made by smaller models alongside it.

Context accumulates. Language model APIs are stateless. To continue a task, the agent sends the system prompt, the tool definitions, the user request, and everything that has happened so far, again, on every step. Tool definitions alone can run to thousands of tokens, and they are billed as input on each call.

Failure has a price. An agent that resolves 80% of tasks has not eliminated 80% of the cost of the work. The other 20% still needs a person, often a more expensive one, and the agent’s spend on those failed attempts is not refunded.

There is also a pricing asymmetry worth knowing. Output tokens typically cost several times more than input tokens, and on models with extended reasoning, the reasoning tokens are billed as output even when the provider does not return the full reasoning text. An agent that “thinks” for 1,500 tokens before a 400-token action is paying output rates for 1,900 tokens.

The cost-per-successful-task formula

The metric we recommend is cost per successful task: the total spend attributable to a workflow divided by the number of tasks that reached a correct outcome.

Cost per successful task = (model spend on all attempts + human review and rescue + allocated platform cost) ÷ tasks completed correctly 

Each term hides decisions that change the result. 

Model spend on all attempts includes failed attempts and automatic retries, not only the runs that succeeded. It also includes secondary models used for routing, moderation, or validation.

Human review and rescue is the cost of people who check flagged outputs or take over tasks the agent could not finish. It is calculated as the share of tasks that need a person, multiplied by the loaded cost of that person’s time on one task.

Allocated platform cost covers what does not scale per call: hosting, vector storage, observability tooling, and the evaluation runs you should be doing before every prompt or model change. Divide the monthly total by monthly task volume.

Tasks completed correctly is the denominator, and it is where many cost reports quietly cheat. A task the agent completed incorrectly, without anyone noticing, is not a success. If you do not measure correctness, you cannot compute this number honestly, which is itself a useful signal about readiness for production.

How agent token usage actually grows

Consider an agent with a 4,000-token starting context that adds 1,500 tokens of history per step (its own output plus the tool result it receives). On step one it sends 4,000 input tokens. On step six it sends 11,500. Across the whole task it is billed for 46,500 input tokens, not the 4,000 a naive estimate would use.

The general shape is simple. If a task starts with a base context of B tokens and each step adds d tokens, the total billed input over n steps is:

Total input tokens = n × B + d × n × (n − 1) ÷ 2

The second term grows with the square of the step count. Doubling the number of steps roughly quadruples the history portion of the bill. This is why a long-horizon agent that runs for 24 steps can bill hundreds of thousands of input tokens for a task that produces a few pages of output.

Prompt caching is the main countermeasure. Most major providers now bill cached input reads at a steep discount, commonly 50% to 90% below the standard input rate, and some charge a premium to write content into the cache. Because agent history is append-only, the unchanged prefix of each request can be served from cache and only the newest tokens are billed at full price. Whether your stack actually achieves this depends on keeping the prefix stable: reordering tool definitions, injecting timestamps near the top of the prompt, or rewriting earlier history breaks the cache match.

Three worked models

We modeled three archetypes that cover most agent deployments: a real-time support agent, a batch document processing pipeline with human review, and a long-running operations agent. The token prices below are illustrative assumptions for a mid-tier model and a larger reasoning-capable model. They are close to the range of current list prices but are not any provider’s quote, so replace them with your own rates before relying on the output.

Model 1: support resolution agent

Each conversation runs six steps: understand the request, retrieve account data, check policy, act, confirm, and respond. The agent bills 46,500 input tokens and 1,800 output tokens per conversation. With the stable system prompt and tool definitions cached, model spend is about $0.12. A small guardrail model adds a tenth of a cent, and the platform allocation adds $0.09.

That puts the agent side at $0.21 per conversation. The 22% of conversations handed to a person add $0.88 on average. The cost per successful task is $1.09, and model tokens account for about 11% of it.

The lesson is uncomfortable for teams focused on model pricing. Moving to a model at half the price saves about six cents per conversation. Resolving ten more conversations in every hundred saves forty.

Model 2: document processing pipeline

Each document runs through one extraction call over three page images, followed by a validation pass on a small text model that checks totals, dates, and required fields. Because documents do not need an answer within seconds, both calls go through a batch interface, which major providers discount by around 50% in exchange for asynchronous processing.

Model spend is under two cents per document. Platform allocation adds two cents. The 12% of documents flagged for review add $0.14, so the cost per successful document is about $0.18, against a manual baseline of $1.50.

Here the dominant cost is review, at 78% of the total. The validation pass is therefore the most valuable component of the pipeline, because a tighter validator that flags fewer false positives cuts review volume directly. Halving the flag rate from 12% to 6% reduces cost per document by 39%.

Model 3: long-running operations agent

This agent handles multi-step internal work such as reconciling records across systems. It runs 24 steps on a larger model with extended reasoning. Each attempt bills 696,000 input tokens and 45,600 output tokens, of which 36,000 are reasoning tokens.

The caching strategy decides the economics. Without caching, one attempt costs $4.62 in model spend. Caching only the stable system prompt brings it to $4.10. Caching the full append-only history, so each step pays full price only for its new tokens, brings it to $1.72. Cache write premiums, where a provider charges them, add about six cents per attempt at these volumes and do not change the conclusion.

With a 70% first-attempt success rate and one automatic retry, the agent averages 1.3 attempts per task. The 9% of tasks that fail twice go to an analyst at $25. Cost per successful task is $5.16 with full history caching and $8.92 without it, against a human baseline of $25.

Cost per call vs cost per successful task

This is the comparison that most budgets skip. The per-call figure is what a quick estimate from a price sheet would produce. The cost per successful task is what the business actually pays. 

Two conclusions follow. First, an agent can be far cheaper than the current process and still cost many times more than a per-call estimate suggests, so the estimate matters for budgeting even when the business case is strong. Second, the ratio grows with loop depth, which is why long-running agents are the ones most often under-budgeted.

Which levers actually move the bill

We changed one assumption at a time and recorded the effect on cost per successful task.

The ranking is stable across the three workloads. Success rate and human cost dominate. Architecture choices such as caching, batching, and loop depth come next. Token price, the number most teams negotiate first, has the smallest effect in short workflows and only a moderate one in long workflows. 

How to reduce cost per successful task

The order matters. Optimizing token spend before you can measure success rate usually saves little and sometimes lowers quality enough to raise the total.

1. Instrument every task end to end. Log each model call with its token counts, cached tokens, reasoning tokens, and a shared task identifier, then record the task outcome: succeeded, escalated, retried, or failed.

2. Establish a correctness check. Build an evaluation set from real tasks and score outcomes, not just whether the agent returned something. Without it, you cannot tell a cheaper configuration from a worse one.

3. Raise the success rate on the most common failures. Review escalated tasks by cause. Missing tool access, ambiguous policies, and poor retrieval are usually cheaper to fix than model quality.

4. Cap loop depth and detect loops. Set a maximum step count per task type and stop the agent when it repeats the same tool call with the same arguments.

5. Make caching work. Keep system prompts, tool definitions, and history stable and at the start of the request, then confirm through billing data that cached token counts are what you expect.

6. Route by difficulty. Send classification, extraction, and validation to smaller models, and reserve larger reasoning models for the steps that need them. Set reasoning effort per step rather than globally.

7. Batch anything that can wait. Overnight reports, document backlogs, and evaluation runs rarely need real-time responses.

8. Prune context. Summarize or drop tool results the agent no longer needs, and retrieve fewer, better documents rather than many marginal ones.

Budget controls that prevent runaway spend

An agent that loops on a confusing input can burn through a day’s budget in minutes. Controls belong in the orchestration layer, not in the prompt, because a model cannot be relied on to stop itself.

Per-task limits. Cap steps, total tokens, and elapsed time for each task type, and end the task with an escalation when any limit is reached.

Per-customer or per-tenant budgets. In multi-tenant products, one heavy user should not be able to consume the margin on everyone else. Meter spend per tenant and throttle or alert at a threshold.

Anomaly alerts. Alert on cost per task, not only total daily spend. A 3x jump in cost per task on stable volume usually means a prompt change broke caching or a tool started returning oversized results.

A kill switch. Keep the ability to pause autonomous actions and route all traffic to people without a deployment. It is cheap to build and expensive to lack.

Comparing against the human baseline

The business case rests on the comparison between cost per successful task and what the same outcome costs today. Build the baseline with the same care as the agent model: loaded hourly cost, handling time per task, and the share of tasks that already need a second touch.

Two adjustments keep the comparison honest. Include the cost of errors that neither the agent nor the reviewer catches, such as refunds, rework, or compliance exposure, on both sides. And account for the people who remain in the loop. In our support model, the human takeover rate is the single largest cost, which means the humans who stay are doing harder work. Their handling time per task may rise, and the baseline should reflect that.

What to measure from day one

Every production agent should report the following for each task type, weekly: task volume, success rate without human help, escalation rate by cause, average steps per task, input, cached, reasoning, and output tokens per task, model spend per task, human review cost per task, and cost per successful task. With those numbers in place, every change to prompts, models, or tools can be judged on the only metric that matters to the business.

Frequently asked questions