# How to throttle outbound API calls with a token bucket

> Keep one process under a documented limit such as 10 requests a second with a client-side token bucket, and check the send log for a second that holds more.

Source: https://voxgig.com/howto/throttle-outbound-calls-with-a-token-bucket

- Audience: api-consumer
- Level: beginner
- Languages: typescript, go, python
- Verified: 2026-11-21
- Published: 2026-11-21

## Short answer

Put one token bucket in front of every call to the API, set just under the documented limit with a burst of one, and share it across the process. sdkgen's ratelimit feature takes `rate` and `burst` in the constructor but lets concurrent calls through together, so wrap `system.fetch` with bottleneck for those. In Go, call `Limiter.Wait` from an oapi-codegen request editor. Then log send times and count the busiest second.

---
## You will need

Node 22 or later, Go 1.24 or later for the Go section, and an API with a documented per-second
limit. npm prints `EBADENGINE` warnings when it installs sdkgen on Node 22, because its packages
declare Node 24, and the scaffold and the generation for this page still completed. The document is `api.json`, a
places API, titled `Geocode`, with one operation, `GET /places`. A stand-in on 127.0.0.1 answers
it and records every request without refusing any, so each row sends all 15 calls.

## Approaches compared

| Approach | When it fits | What it costs you | When to pick something else |
| --- | --- | --- | --- |
| [a fixed sleep between calls](https://nodejs.org/api/timers.html#timerspromisessettimeoutdelay-value-options) | One serial loop and a limit well beyond what you need | The sleep adds to every call's latency, and concurrent callers ignore it | Calls come from more than one place in the code |
| [bottleneck `minTime` and `maxConcurrent`](https://github.com/SGrondin/bottleneck) | Node code with concurrent callers, wrapped around one function every call passes through | A dependency last released in 2019, and gaps that can run several milliseconds short of `minTime` | You need the limit in more than one language |
| [Fern custom fetcher](https://buildwithfern.com/learn/sdks/current/generators/typescript/dynamic-authentication) | A Fern TypeScript SDK, where `allowCustomFetcher` in `sdk-config.yml` adds a `fetcher` that wraps every request | No limiter of its own, so you bring one, and generating needs a Fern login or Docker | You cannot change the SDK's generator configuration |
| [oapi-codegen request editor with x/time/rate](https://github.com/oapi-codegen/oapi-codegen#generating-api-clients) | A Go client, where an editor runs before each request and can block | You wire the limiter into every client, and its long-run rate allows a short burst | The client is not Go |
| [Python ratelimit and pyrate-limiter](https://pyratelimiter.readthedocs.io/en/latest/) | Python code, with a decorator or a limiter object around the call | A decorator covers one function, so every other call path needs its own | The calls go through a generated client you cannot decorate |
| [sdkgen ratelimit feature](https://github.com/voxgig/sdkgen/blob/main/docs/reference/features.md) | Serial calls through one generated client, with the same option block in each target | Concurrent calls on one client pass together, and the default burst equals the rate | Calls overlap, or several clients share one key |

Voxgig maintains sdkgen. The table holds six options and recommends none of them.

The sdkgen feature costs one line in the constructor and holds serial calls, but only those.
oapi-codegen generates no limiter, and the editor it does generate gives `x/time/rate` a place
that every request passes. Fern documents no limiter either, so its fetcher is where bottleneck
would go.

## Turn on the ratelimit feature with a burst of one

`generate.sh` scaffolds the project with the feature and generates the TypeScript SDK.

```bash title="generate.sh"
cp api.json "$work/"
cd "$work"
npx --yes @voxgig/create-sdkgen@0.31.9 geocode --def api.json --target ts --feature ratelimit
cd geocode-sdk/.sdk
npm run generate
```

The feature is off until the constructor activates it. The
[feature reference](https://github.com/voxgig/sdkgen/blob/main/docs/reference/features.md) gives
`rate` in tokens per second and a `burst` that defaults to `rate`, and calls it a budget per client
instance.

```js title="throttle.mjs"
export const limitedClient = (base, fetch, ratelimit) => new GeocodeSDK({
  base,
  system: { fetch },
  feature: { ratelimit: { active: true, ...ratelimit } },
})
```

The `fetch` passed in `system` records the moment each request leaves, after the limiter has let it
go. `throttle.mjs` makes 15 calls per row and counts the most sends in any one-second window.

```bash
node throttle.mjs
```

```text output
15 calls to GET /places, documented limit 10 per second
  scenario                                   sent  busiest second
  no limiter                                 15    15, over the limit
  sdkgen ratelimit { rate: 9 }               15    15, over the limit
  sdkgen ratelimit { rate: 9, burst: 1 }     15    10 or fewer
    the same, a client per call              15    15, over the limit
    the same, 15 calls at once               15    15, over the limit
  bottleneck { minTime: 110 }, 15 at once    15    10 or fewer
  a fixed 110 ms sleep between calls         15    10 or fewer
```

`rate: 9` alone starts with nine tokens, sends them at once, then refills, so its first second held
all 15. A limit stated per second is a window, and a bucket as deep as the rate can put up to twice
the rate into one. The test replays `rate: 10` on a clock it owns, through the feature's `now` and
`sleep` options, and counts 19 in one second.

Set `burst: 1`, and set the rate below the limit. At `rate: 10` the bucket spaces serial calls
exactly 100 ms apart, which leaves no room for timer jitter, and sends here came as close as 96 ms.
Every paced row runs at nine a second, so each verdict has about 100 ms of headroom.

## Hold concurrent calls with bottleneck in `system.fetch`

Fifteen calls started together on one limited client all went out in the same second. In 4.44.3
the feature checks the bucket, sees it empty, and every waiting call sleeps for the same one token,
then proceeds. The Go and Python targets have the same acquire step and no lock. In a scratch run,
15 goroutines on one Go client went out within 102 ms, and `go test -race` reported a data race in
the feature. Fifteen Python threads went out in clumps, and one run of three held 11 in a second.
sdkgen 4.45.0 generated identical source for all three targets and sent the 15 TypeScript calls
within 100 ms.

The SDK takes its transport in `system.fetch`, so a limiter that queues goes there.

```js title="throttle.mjs"
  'bottleneck { minTime: 110 }, 15 at once': async (base) => {
    const limiter = new Bottleneck({ maxConcurrent: 1, minTime: 110 })
    const log = sendLog()
    const client = new GeocodeSDK({ base, system: { fetch: limiter.wrap(log.fetch) } })
    await Promise.all(Array.from({ length: CALLS }, () => client.Place().list()))
    return log.sends
  },
```

`minTime: 110` is nine a second, not 10. Bottleneck's own default is 0, so 110 is this page's
choice. With `minTime: 100`, the shortest gap in each of three scratch runs here was under 100 ms,
97.2 ms at the lowest, so 11 calls fitted into one second.

## Wait on `x/time/rate` in an oapi-codegen request editor

oapi-codegen writes a Go client with no limiter. Each method takes `reqEditors`, and
`WithRequestEditorFn` adds one for every call; an editor that returns after
[`Limiter.Wait`](https://pkg.go.dev/golang.org/x/time@v0.14.0/rate) holds the request until a
token is free.

```go title="go/main.go"
// waitFor returns a request editor that blocks until the limiter allows a request.
func waitFor(l *rate.Limiter) geocode.RequestEditorFn {
	return func(ctx context.Context, req *http.Request) error {
		return l.Wait(ctx)
	}
}
```

```bash
cd go && go run .
```

```text output
15 calls to GET /places through the oapi-codegen client, limit 10 per second
  scenario                                     sent  busiest second
  rate.NewLimiter(10, 10), one client          15    15, over the limit
  rate.Every(110ms), burst 1, one client       15    10 or fewer
    the same, 15 goroutines at once            15    10 or fewer
    the same, a client and limiter per call    15    15, over the limit
rate.NewLimiter(10, 1) on a controlled clock, the second request let go 30 ms late
  sends at ms: 0 130 200 300 400 500 600 700 800 900 1000 1100
  busiest second: 11
```

The package documentation says a limiter is safe for simultaneous use by multiple goroutines, and
15 at once stayed under the limit. The last block replays `NewLimiter(10, 1)` on a clock the
program sets. The second request went 30 ms late, and the third went 70 ms after it. The limiter
booked the third token from the second one's reserved time, so the one-second window from 130 ms
held 11.[^1] The paced rows use `rate.Every(110ms)` for that reason.

## Check it worked

The test runs each row against a fresh stand-in and asserts on the busiest second of the send log.

```js title="throttle.test.mjs"
test('sdkgen ratelimit at rate 9, burst 1, holds serial calls under the limit', async () => {
  const r = await measure('sdkgen ratelimit { rate: 9, burst: 1 }')
  assert.equal(r.sent, CALLS)
  assert.equal(r.arrived, CALLS)
  assert.ok(r.most <= LIMIT, `busiest second held ${r.most}`)
})
```

```bash
node --test throttle.test.mjs
```

```text output
1..11
# tests 11
# suites 0
# pass 11
# fail 0
# cancelled 0
# skipped 0
# todo 0
# duration_ms 5236.241507
```

`# pass 11` holds the verdicts in the demo output, including the concurrent row going over, and the
two counts from the controlled clock. `go test ./...` in `go/` asserts the Go rows and the 70 ms
gap.

## When it goes wrong

The busiest second holds 15 although each client has a limiter. A client built per request, or
per worker, brings a full bucket with it, so the rate multiplies by the number of clients. Build
one client or one limiter for the process, and pass it to every caller.

The API answers 429 although the send log shows no second over 10. The server counts arrivals,
and network delay can close a gap between two sends. Leave headroom below the documented limit,
as the paced rows do, and read the API's own headers where it sends them.

A Fern TypeScript client goes over the limit after a 429. Fern's
[retries page](https://buildwithfern.com/learn/sdks/current/deep-dives/retries-with-backoff) says
its SDKs retry 429 up to two times by default, and a retry is a request. Fern's documentation does
not say whether those retries pass back through a custom `fetcher`, so set `maxRetries: 0` and
retry inside the fetcher, behind the limiter. Fern's [pricing page](https://buildwithfern.com/pricing)
puts TypeScript and Python SDKs on the Free plan.

## When not to do this

Do not rely on sdkgen's ratelimit feature for calls that overlap. It holds serial calls on one
client at `burst: 1`, and it let 15 concurrent calls through within about 100 ms. Put bottleneck in
`system.fetch` instead, or keep the calls serial.

Do not add a client-side bucket when several processes share the key. Each process gets its own
bucket, and the sum goes over. A shared limiter in Redis is the answer there.

Pick oapi-codegen with `x/time/rate` when the caller is Go and calls run in goroutines, because
that limiter is safe for concurrent use and one editor covers every method. Pick Fern when a
hosted pipeline owns your SDKs and the team already has a limiter to put in its fetcher.

## Related how-tos

- [Read RateLimit headers and slow down before you get a 429](/howto/read-ratelimit-headers-and-pace-before-429)

- [Share one rate limiter across many workers with Redis](/howto/share-a-rate-limiter-across-workers-with-redis)

- [Choose between 429 and 503 when you throttle a request](/howto/choose-429-or-503-when-throttling)

- [Rate limit by API key and tenant, not client IP](/howto/key-rate-limits-by-api-key-and-tenant)

- [Inject latency and failures into SDK tests](/howto/inject-latency-and-failures-into-sdk-tests)

## Last verified

Verified 2026-11-21 against Node 22.23.5, Go 1.24.7, `@voxgig/create-sdkgen` 0.31.9,
`@voxgig/sdkgen` 4.44.3, `@voxgig/apidef` 8.32.4, bottleneck 2.19.5, oapi-codegen v2.8.2 and
`golang.org/x/time` v0.14.0. Every output block is what the command preceding it printed. sdkgen
and oapi-codegen ran on `api.json`, and their output is committed. The Go and Python targets of the
same project, and sdkgen 4.45.0, ran once on the concurrent case and are not committed. The gap
figures in prose come from scratch runs on this machine. Fern was not run, and its fetcher option and the
plan each language needs are read from its documentation and its pricing page.

[^1]: The package documentation for
[`x/time/rate`](https://pkg.go.dev/golang.org/x/time@v0.14.0/rate) states its guarantee in one
qualified sentence: informally, in any large enough time interval, the limiter limits the rate to r tokens
per second. The qualifier matters, because a late wake is repaid out of the next
token's wait, and the repayment can fall inside the same second as the late request.