An agent asked to find one parcel searches, gets nothing, and searches again with the name reworded. Every call sends the whole transcript again, so every call costs more than the last. When its step limit ends the run, it has made 25 model calls and spent 52,405 tokens on a question that takes 248 when it can be answered.
What you get
You will end up with an agent loop that gives each task a token allowance, stops between calls when it is gone, and returns what it found with the reason. This is for you if a looping agent can spend money nobody budgeted.
Short answer
Give each agent task a token allowance by building one sdkgen client per task, with the cost
feature reading usage.total_tokens and its budget set to the allowance. The client refuses
the next model call with cost_budget once the allowance is spent, and the loop turns that
refusal into a partial result with a reason. The check runs between calls, so hold back one
call’s worth of tokens, and keep a second allowance per tenant.
You will need
Node 22 or later, Python 3 for the loop with no SDK, and a model API whose responses report token
usage. Verified 2026-11-19 against Node 22.23.5, Python 3.13.18, @voxgig/create-sdkgen 0.31.9,
and @voxgig/sdkgen 4.44.3. Reading and pricing the usage fields is covered in
account for tokens and cost per agent task.
The model here is a stand-in on the loopback interface. Its usage object has the fields the Responses API documents, and its counts are a simplification: one input token per four characters of input.
Approaches compared
| Approach | When it fits | What it costs you | When to pick something else |
|---|---|---|---|
| Fern custom fetcher | An SDK you generate with Fern, whose users should bring their own request policy | A producer setting, allowCustomFetcher, plus a wrapper each user writes to count and refuse | The SDK’s producer left the option off |
| LangGraph recursion limit | A graph that must not run forever, whatever a step costs | It counts super-steps, defaults to 10,007 of them, and ends the run with an exception | The cost of a step grows with the transcript |
| LiteLLM budgets | Many services behind one gateway, capped per key, team, or customer | A proxy and a database to run, budgets in dollars, and no view of the task | The limit belongs to one task in one process |
OpenAI Agents SDK max_turns | An agent on that SDK that needs a ceiling on turns | It counts turns, not tokens, and ends in MaxTurnsExceeded unless an error_handlers entry supplies a final output | Turns differ widely in cost |
sdkgen cost feature budget | A model API client generated by sdkgen, built once per task | A cutoff: the last admitted call can pass the allowance, and each client counts alone | The model SDK is not sdkgen’s, where a counter in the loop does the same |
| Speakeasy SDK hooks | An SDK you generate with Speakeasy on a Business or Enterprise plan | Hooks are compiled into the SDK by its producer, and limited to those plans | You only consume the SDK and cannot add a hook |
Voxgig maintains sdkgen. It is one of six options here, not the recommendation.
Framework limits count steps, and a step is not a unit of cost: the last call of the run below costs 71 times the first. A proxy such as LiteLLM can refuse a call before it is sent, but it budgets keys, teams, and customers, not tasks. A cutoff between calls needs no framework and no proxy, but it lets the last call through.
Give each task its own client and allowance
The cost feature has to be in the generated client, so name it when you scaffold.
npx --yes @voxgig/create-sdkgen@0.31.9 quill --def api.json --target ts --feature cost
cd quill-sdk/.sdk
npm run generate
Then build one client per task. path points at the usage figure in the response body, perUnit
prices one token as one unit, and budget with onBudget: 'deny' is the allowance.
const client = new QuillSDK({
base,
feature: {
cost: {
active: true,
currency: 'tokens',
// Read each call's price from the response body, one unit per token.
path: 'usage.total_tokens',
perUnit: 1,
// A response with no usage counts as a worst-case call, not as nothing.
unit: WORST_CALL,
// The budget is checked against tokens already spent, so the last call
// admitted can pass it. Hold back one call's worth to end under the allowance.
...(limited ? { budget: allowance - holdBack, onBudget: 'deny' } : {}),
// Called after every model call with the tokens it was charged.
sink: (record) => {
costs.push(record.amount)
tenant?.add(record.amount)
},
},
},
})
The budget belongs to the client instance, which is what makes it per task. sdkgen’s reference
describes it as a cutoff scoped to one client, not a hard cap.1 currency is only a label, so
the feature that counts dollars elsewhere counts tokens here. After every call
client._cost.budget reports the limit, the tokens spent, and what remains.
Turn the refusal into a partial result
Once the allowance is spent, the client refuses the next call with cost_budget before sending
it. The loop catches that one code and returns, and lets every other error through. It keeps its
step limit as a second stop, and every exit goes through one function that reports the status,
the reason, the tokens, and the tool results so far.
for (let step = 1; step <= maxSteps; step++) {
if (tenant?.spent) return stop('stopped', 'tenant_budget')
let reply
try {
reply = (await client.Response().create({ model: 'quill-small', input, max_output_tokens: 256 })).data()
} catch (err) {
// The client refused to start the call: the allowance is gone. Anything else is a real failure.
if (err.code === 'cost_budget') return stop('stopped', 'token_budget')
throw err
}
if (!reply.tool_call) return stop('done', null, reply.output_text)
node demo.mjs
a task the agent can finish: "Where is order 1182?"
allowance of 6000 tokens done 2 calls 248 tokens, 0 over
answer: Order 1182 is out for delivery, today before 18:00.
a task it cannot: "Where is the parcel for Dana Whitfield?"
step limit of 25 alone stopped: step_limit 25 calls 52405 tokens
its first call cost 58 tokens and its last cost 4135
allowance of 6000 tokens stopped: token_budget 9 calls 6639 tokens, 639 over
6000 with 1500 held back stopped: token_budget 8 calls 5222 tokens, 0 over
partial result: 8 tool results, the last for "Whitfield Dana"
ten runs of the first task for one tenant, each with its own allowance of 6000
per-task allowance alone 10 done, 0 stopped, 2480 tokens for the tenant
tenant allowance of 1000 4 done, 6 stopped, 1045 tokens for the tenant
The allowance ended the runaway task after nine calls, at about an eighth of what the step limit let through. It also ran 639 tokens over. The ninth call was admitted with 5,222 spent, and the feature learns what a call costs only when the response arrives. Holding back 1,500 tokens, a little more than that call, ended the run under the allowance. Either way the caller gets the tool results gathered so far and a reason, not an exception from the middle of a step.
Add an allowance for the tenant
A per-task allowance says nothing about the sum. Ten runs for one tenant each finished far below
6,000 tokens and together spent 2,480 that no limit covered. Keep a second counter per tenant,
feed it from the sink of every task’s client, and check it between calls as well.
export function tenantAllowance(name, limit) {
let used = 0
return {
name,
limit,
add(tokens) { used += tokens },
get used() { return used },
get spent() { return used >= limit },
}
}
With a tenant allowance of 1,000, the fifth run stopped after one call and the rest made none. That counter lives in one process. Tenants served by several processes need a shared store.
Keep the same count in a loop with no SDK
The counter does not need a generated client. Any SDK that returns the usage object works, whichever generator produced it: charge the usage after each call and ask before the next.
for _ in range(max_steps):
# Between calls: the only place a loop can stop without losing a response it paid for.
if budget.spent:
return stop("stopped", "token_budget")
reply = model({"model": "quill-small", "input": items, "max_output_tokens": 256})
calls += 1
budget.charge(reply["usage"])
Between calls is the one place a loop can stop without throwing away a response it has paid for. The counter itself refuses a usage object it cannot read, so a changed response shape fails loudly and is never counted as zero.
def charge(self, usage):
"""Add one response's usage. A usage object it cannot read is an error, never a zero."""
tokens = usage.get("total_tokens") if isinstance(usage, dict) else None
if isinstance(tokens, bool) or not isinstance(tokens, int) or tokens < 0:
raise ValueError(f"usage carries no usable total_tokens: {usage!r}")
self.used += tokens
return tokens
@property
def spent(self):
"""True once another call may not start. Checked between calls, so it is a cutoff."""
return self.used >= self.allowance - self.hold_back
python3 agent.py
own loop, no SDK: "Where is the parcel for Dana Whitfield?"
allowance of 6000 tokens stopped: token_budget 9 calls 6639 tokens, 639 over
6000 with 1500 held back stopped: token_budget 8 calls 5222 tokens, 0 over
Enforce the allowance inside a Speakeasy or Fern SDK
Speakeasy and Fern document no budget option. Speakeasy’s
extension list has none, and its
mechanism for this is an SDK hook. beforeRequest may throw to stop a request, and
afterSuccess receives the response before it is deserialized, which is where the usage can be
read. The hook context carries the operation id and the SDK’s options, so the allowance has to be
found through those. Hooks are registered in src/hooks/registration.ts, a file generated once
and never overwritten, so they belong to whoever generates the SDK. They are available to Business
and Enterprise users only. If you consume a Speakeasy-generated SDK, count in the loop.
The README of Fern’s open source repository says that project
is no longer maintained and points to the hosted service. Its current sdk-config.yml format has
no budget key. It does have a setting that makes the generated TypeScript client accept a
fetcher option, which wraps every HTTP request:
targets:
- language: typescript
generation:
allowCustomFetcher: true
Fern documents the fetcher as middleware for authentication. A fetcher that refuses when the allowance is gone, and counts usage on the way back, puts the check inside the client by the same route. The SDK’s producer has to switch the setting on, and each user supplies the function.
Check it worked
Run the suite. It checks calls and tokens against the stand-in’s own counts.
node --test budget.test.mjs
1..13
# tests 13
# suites 0
# pass 13
# fail 0
# cancelled 0
# skipped 0
# todo 0
# pass 13. One test asserts that the runaway task stops with token_budget after 9 calls and
6639 tokens, and that the tenth call never reached the model. Another runs the Python loop and
expects the same two numbers.
When it goes wrong
The allowance never trips. The responses carried no usage.total_tokens, so each call was priced
at zero. Set unit to a worst-case call, as the client on this page does: it applies only when
the body figure is missing.
An allowance of zero stops nothing. The feature reads budget: 0 as no limit, so settle that case
before the first call.
A network failure is reported as a spent budget. The loop caught every error. Match the
cost_budget code and throw the rest again.
The run ends over the allowance. One call can cost more than what remained. Hold back the size of
your largest call, and cap its output with max_output_tokens.
When not to do this
Do not present a between-calls cutoff as a cap. One oversized call passes it, and the tokens are spent by the time the count arrives.
Do not switch model clients to sdkgen for this counter. The Python loop here is the same allowance in one small
class, with any SDK. sdkgen’s budget earns its place when the model client is generated by
sdkgen already.
Do not use sdkgen’s opt-in feature where the stop must not be optional. A Speakeasy hook is
compiled into every copy of the SDK you publish, and sdkgen’s feature waits for each caller to
activate it. Fern’s fetcher option is the better fit when each user should bring a policy of
their own, because it hands them every request.
Do not rely on a step limit for cost. LangGraph’s default allows 10,007 super-steps.2
Do not skip the proxy when many processes share the spend. LiteLLM refuses a call across all of them, provided its database is connected.3
Related how-tos
Last verified
Verified 2026-11-19 against Node 22.23.5, Python 3.13.18, @voxgig/create-sdkgen 0.31.9, and
@voxgig/sdkgen 4.44.3. Every output block is what the command preceding it printed. No request
reached a model provider: the model is a stand-in, and its token counts are a simplification.
Speakeasy and Fern were not run; their options come from their documentation. LangGraph, LiteLLM,
and the OpenAI Agents SDK were not run either.
Footnotes
-
The features reference says the budget is per client instance, so two processes get two budgets and neither knows about the other. It stops this client, the same passage says, not your account. It also calls the cutoff the useful shape for agent traffic, on the grounds that such traffic is bursty, repetitive, and prone to tight retry loops. ↩︎ Back to text
-
The LangGraph documentation dates the figure: 10,000 steps between versions 1.0.6 and 1.1.3, and 10,007 super-steps starting in 1.1.4. The page does not say what the seven are for. The result is a prime number. ↩︎ Back to text
-
LiteLLM’s budgets page says every budget on it is enforced against spend read from the database, so none of them caps anything on a deployment without one. The global budget, it adds, fails open there: the check is skipped and requests keep being served past the limit. A warning is logged once at startup. ↩︎ Back to text