# How to set a per-task token budget that stops a runaway agent

> Give each agent task a token allowance, stop between model calls when it is gone, and return what the task found with the reason it stopped.

Source: https://voxgig.com/howto/set-a-per-task-token-budget

- Audience: agent-builder
- Level: intermediate
- Languages: python, typescript
- Verified: 2026-11-19
- Published: 2026-11-19

## Short answer

Give each agent task a token allowance by building one sdkgen client per task, with the `cost` feature reading `usage.total_tokens` and its `budget` set to the allowance. The client refuses the next model call with `cost_budget` once the allowance is spent, and the loop turns that refusal into a partial result with a reason. The check runs between calls, so hold back one call's worth of tokens, and keep a second allowance per tenant.

---
## You will need

Node 22 or later, Python 3 for the loop with no SDK, and a model API whose responses report token
usage. Verified 2026-11-19 against Node 22.23.5, Python 3.13.18, `@voxgig/create-sdkgen` 0.31.9,
and `@voxgig/sdkgen` 4.44.3. Reading and pricing the usage fields is covered in
[account for tokens and cost per agent task](/howto/account-for-tokens-and-cost-per-task).

The model here is a stand-in on the loopback interface. Its usage object has the fields the
[Responses API](https://developers.openai.com/api/reference/resources/responses) documents, and its
counts are a simplification: one input token per four characters of input.

## Approaches compared

| Approach | When it fits | What it costs you | When to pick something else |
| --- | --- | --- | --- |
| [Fern custom fetcher](https://buildwithfern.com/learn/sdks/current/generators/typescript/dynamic-authentication) | An SDK you generate with Fern, whose users should bring their own request policy | A producer setting, `allowCustomFetcher`, plus a wrapper each user writes to count and refuse | The SDK's producer left the option off |
| [LangGraph recursion limit](https://docs.langchain.com/oss/python/langgraph/graph-api#recursion-limit) | A graph that must not run forever, whatever a step costs | It counts super-steps, defaults to 10,007 of them, and ends the run with an exception | The cost of a step grows with the transcript |
| [LiteLLM budgets](https://docs.litellm.ai/docs/proxy/users) | Many services behind one gateway, capped per key, team, or customer | A proxy and a database to run, budgets in dollars, and no view of the task | The limit belongs to one task in one process |
| [OpenAI Agents SDK `max_turns`](https://openai.github.io/openai-agents-python/running_agents/#the-agent-loop) | An agent on that SDK that needs a ceiling on turns | It counts turns, not tokens, and ends in `MaxTurnsExceeded` unless an `error_handlers` entry supplies a final output | Turns differ widely in cost |
| [sdkgen `cost` feature budget](https://github.com/voxgig/sdkgen/blob/main/docs/reference/features.md#the-budget) | A model API client generated by sdkgen, built once per task | A cutoff: the last admitted call can pass the allowance, and each client counts alone | The model SDK is not sdkgen's, where a counter in the loop does the same |
| [Speakeasy SDK hooks](https://www.speakeasy.com/docs/sdks/customize/code/sdk-hooks) | An SDK you generate with Speakeasy on a Business or Enterprise plan | Hooks are compiled into the SDK by its producer, and limited to those plans | You only consume the SDK and cannot add a hook |

Voxgig maintains sdkgen. It is one of six options here, not the recommendation.

Framework limits count steps, and a step is not a unit of cost: the last call of the run below
costs 71 times the first. A proxy such as LiteLLM can refuse a call before it is sent, but it
budgets keys, teams, and customers, not tasks. A cutoff between calls needs no framework and no
proxy, but it lets the last call through.

## Give each task its own client and allowance

The `cost` feature has to be in the generated client, so name it when you scaffold.

```bash title="generate.sh"
npx --yes @voxgig/create-sdkgen@0.31.9 quill --def api.json --target ts --feature cost
cd quill-sdk/.sdk
npm run generate
```

Then build one client per task. `path` points at the usage figure in the response body, `perUnit`
prices one token as one unit, and `budget` with `onBudget: 'deny'` is the allowance.

```js title="agent.mjs"
  const client = new QuillSDK({
    base,
    feature: {
      cost: {
        active: true,
        currency: 'tokens',
        // Read each call's price from the response body, one unit per token.
        path: 'usage.total_tokens',
        perUnit: 1,
        // A response with no usage counts as a worst-case call, not as nothing.
        unit: WORST_CALL,
        // The budget is checked against tokens already spent, so the last call
        // admitted can pass it. Hold back one call's worth to end under the allowance.
        ...(limited ? { budget: allowance - holdBack, onBudget: 'deny' } : {}),
        // Called after every model call with the tokens it was charged.
        sink: (record) => {
          costs.push(record.amount)
          tenant?.add(record.amount)
        },
      },
    },
  })
```

The budget belongs to the client instance, which is what makes it per task. sdkgen's reference
describes it as a cutoff scoped to one client, not a hard cap.[^1] `currency` is only a label, so
the feature that counts dollars elsewhere counts tokens here. After every call
`client._cost.budget` reports the limit, the tokens spent, and what remains.

## Turn the refusal into a partial result

Once the allowance is spent, the client refuses the next call with `cost_budget` before sending
it. The loop catches that one code and returns, and lets every other error through. It keeps its
step limit as a second stop, and every exit goes through one function that reports the status,
the reason, the tokens, and the tool results so far.

```js title="agent.mjs"
  for (let step = 1; step <= maxSteps; step++) {
    if (tenant?.spent) return stop('stopped', 'tenant_budget')
    let reply
    try {
      reply = (await client.Response().create({ model: 'quill-small', input, max_output_tokens: 256 })).data()
    } catch (err) {
      // The client refused to start the call: the allowance is gone. Anything else is a real failure.
      if (err.code === 'cost_budget') return stop('stopped', 'token_budget')
      throw err
    }
    if (!reply.tool_call) return stop('done', null, reply.output_text)
```

```bash
node demo.mjs
```

```text output
a task the agent can finish: "Where is order 1182?"
  allowance of 6000 tokens   done                    2 calls    248 tokens, 0 over
  answer: Order 1182 is out for delivery, today before 18:00.
a task it cannot: "Where is the parcel for Dana Whitfield?"
  step limit of 25 alone     stopped: step_limit    25 calls  52405 tokens
  its first call cost 58 tokens and its last cost 4135
  allowance of 6000 tokens   stopped: token_budget   9 calls   6639 tokens, 639 over
  6000 with 1500 held back   stopped: token_budget   8 calls   5222 tokens, 0 over
  partial result: 8 tool results, the last for "Whitfield Dana"
ten runs of the first task for one tenant, each with its own allowance of 6000
  per-task allowance alone   10 done, 0 stopped, 2480 tokens for the tenant
  tenant allowance of 1000   4 done, 6 stopped, 1045 tokens for the tenant
```

The allowance ended the runaway task after nine calls, at about an eighth of what the step limit
let through. It also ran 639 tokens over. The ninth call was admitted with 5,222 spent, and the
feature learns what a call costs only when the response arrives. Holding back 1,500 tokens, a
little more than that call, ended the run under the allowance. Either way the caller gets the
tool results gathered so far and a reason, not an exception from the middle of a step.

## Add an allowance for the tenant

A per-task allowance says nothing about the sum. Ten runs for one tenant each finished far below
6,000 tokens and together spent 2,480 that no limit covered. Keep a second counter per tenant,
feed it from the `sink` of every task's client, and check it between calls as well.

```js title="agent.mjs"
export function tenantAllowance(name, limit) {
  let used = 0
  return {
    name,
    limit,
    add(tokens) { used += tokens },
    get used() { return used },
    get spent() { return used >= limit },
  }
}
```

With a tenant allowance of 1,000, the fifth run stopped after one call and the rest made none.
That counter lives in one process. Tenants served by several processes need a shared store.

## Keep the same count in a loop with no SDK

The counter does not need a generated client. Any SDK that returns the usage object works,
whichever generator produced it: charge the usage after each call and ask before the next.

```python title="agent.py"
    for _ in range(max_steps):
        # Between calls: the only place a loop can stop without losing a response it paid for.
        if budget.spent:
            return stop("stopped", "token_budget")
        reply = model({"model": "quill-small", "input": items, "max_output_tokens": 256})
        calls += 1
        budget.charge(reply["usage"])
```

Between calls is the one place a loop can stop without throwing away a response it has paid for.
The counter itself refuses a usage object it cannot read, so a changed response shape fails loudly
and is never counted as zero.

```python title="budget.py"
    def charge(self, usage):
        """Add one response's usage. A usage object it cannot read is an error, never a zero."""
        tokens = usage.get("total_tokens") if isinstance(usage, dict) else None
        if isinstance(tokens, bool) or not isinstance(tokens, int) or tokens < 0:
            raise ValueError(f"usage carries no usable total_tokens: {usage!r}")
        self.used += tokens
        return tokens

    @property
    def spent(self):
        """True once another call may not start. Checked between calls, so it is a cutoff."""
        return self.used >= self.allowance - self.hold_back
```

```bash
python3 agent.py
```

```text output
own loop, no SDK: "Where is the parcel for Dana Whitfield?"
  allowance of 6000 tokens   stopped: token_budget   9 calls   6639 tokens, 639 over
  6000 with 1500 held back   stopped: token_budget   8 calls   5222 tokens, 0 over
```

## Enforce the allowance inside a Speakeasy or Fern SDK

Speakeasy and Fern document no budget option. Speakeasy's
[extension list](https://www.speakeasy.com/docs/speakeasy-reference/extensions) has none, and its
mechanism for this is an SDK hook. `beforeRequest` may throw to stop a request, and
`afterSuccess` receives the response before it is deserialized, which is where the usage can be
read. The hook context carries the operation id and the SDK's options, so the allowance has to be
found through those. Hooks are registered in `src/hooks/registration.ts`, a file generated once
and never overwritten, so they belong to whoever generates the SDK. They are available to Business
and Enterprise users only. If you consume a Speakeasy-generated SDK, count in the loop.

The README of Fern's [open source repository](https://github.com/fern-api/fern) says that project
is no longer maintained and points to the hosted service. Its current `sdk-config.yml` format has
no budget key. It does have a setting that makes the generated TypeScript client accept a
`fetcher` option, which wraps every HTTP request:

```yaml
targets:
  - language: typescript
    generation:
      allowCustomFetcher: true
```

Fern documents the fetcher as middleware for authentication. A fetcher that refuses when the
allowance is gone, and counts usage on the way back, puts the check inside the client by the same
route. The SDK's producer has to switch the setting on, and each user supplies the function.

## Check it worked

Run the suite. It checks calls and tokens against the stand-in's own counts.

```bash
node --test budget.test.mjs
```

```text output
1..13
# tests 13
# suites 0
# pass 13
# fail 0
# cancelled 0
# skipped 0
# todo 0
```

`# pass 13`. One test asserts that the runaway task stops with `token_budget` after `9` calls and
`6639` tokens, and that the tenth call never reached the model. Another runs the Python loop and
expects the same two numbers.

## When it goes wrong

The allowance never trips. The responses carried no `usage.total_tokens`, so each call was priced
at zero. Set `unit` to a worst-case call, as the client on this page does: it applies only when
the body figure is missing.

An allowance of zero stops nothing. The feature reads `budget: 0` as no limit, so settle that case
before the first call.

A network failure is reported as a spent budget. The loop caught every error. Match the
`cost_budget` code and throw the rest again.

The run ends over the allowance. One call can cost more than what remained. Hold back the size of
your largest call, and cap its output with `max_output_tokens`.

## When not to do this

Do not present a between-calls cutoff as a cap. One oversized call passes it, and the tokens are
spent by the time the count arrives.

Do not switch model clients to sdkgen for this counter. The Python loop here is the same allowance in one small
class, with any SDK. sdkgen's `budget` earns its place when the model client is generated by
sdkgen already.

Do not use sdkgen's opt-in feature where the stop must not be optional. A Speakeasy hook is
compiled into every copy of the SDK you publish, and sdkgen's feature waits for each caller to
activate it. Fern's `fetcher` option is the better fit when each user should bring a policy of
their own, because it hands them every request.

Do not rely on a step limit for cost. LangGraph's default allows 10,007 super-steps.[^2]

Do not skip the proxy when many processes share the spend. LiteLLM refuses a call across all of
them, provided its database is connected.[^3]

## Related how-tos

- [Account for tokens and cost per agent task](/howto/account-for-tokens-and-cost-per-task)

- [Stop a runaway agent loop from exhausting your MCP server](/howto/contain-a-runaway-agent-loop)

- [Measure the token cost of an agent task against your API](/howto/measure-token-cost-per-agent-task)

- [Wire an API client into the OpenAI Agents SDK](/howto/wire-an-api-client-into-openai-agents-sdk)

- [Build a stateful API agent with LangGraph](/howto/stateful-api-agent-with-langgraph)

- [Cache tool results across the steps of one agent run](/howto/cache-tool-results-across-agent-steps)

## Last verified

Verified 2026-11-19 against Node 22.23.5, Python 3.13.18, `@voxgig/create-sdkgen` 0.31.9, and
`@voxgig/sdkgen` 4.44.3. Every output block is what the command preceding it printed. No request
reached a model provider: the model is a stand-in, and its token counts are a simplification.
Speakeasy and Fern were not run; their options come from their documentation. LangGraph, LiteLLM,
and the OpenAI Agents SDK were not run either.

[^1]: The [features reference](https://github.com/voxgig/sdkgen/blob/main/docs/reference/features.md#the-budget)
says the budget is per client instance, so two processes get two budgets and neither knows about
the other. It stops this client, the same passage says, not your account. It also calls the
cutoff the useful shape for agent traffic, on the grounds that such traffic is bursty,
repetitive, and prone to tight retry loops.

[^2]: The LangGraph [documentation](https://docs.langchain.com/oss/python/langgraph/graph-api#recursion-limit)
dates the figure: 10,000 steps between versions 1.0.6 and 1.1.3, and 10,007 super-steps starting
in 1.1.4. The page does not say what the seven are for. The result is a prime number.

[^3]: LiteLLM's [budgets page](https://docs.litellm.ai/docs/proxy/users) says every budget on it is
enforced against spend read from the database, so none of them caps anything on a deployment
without one. The global budget, it adds, fails open there: the check is skipped and requests keep
being served past the limit. A warning is logged once at startup.