How-to › Make calls that survive failure

How to share one rate limiter across many workers with Redis#

Keep twenty workers under one API key's quota by moving the window into Redis, timed by Redis and not by a worker's clock, and measure what reaches the API.

Audience
Platform team
Level
advanced
Topic
Set and respect rate limits
Languages
TypeScript, Go and Python
Verified

Twenty workers share one API key with a quota of 50 requests a second. Each worker paces itself at 50 a second, so the key sends 1000 and the API answers most of them with 429. Each worker’s limiter sees only that worker’s traffic, so none of them knows the quota is gone.

What you get

You will end up with one limiter per API key, held in Redis, that every worker waits on before it sends, wired into three generated clients. A run of twenty workers reaches the API at the quota each second with no 429. This is for you if a fleet shares one key.

Short answer

Keep one counter per API key in Redis. Count each request with a script that runs INCR and gives the counter its expiry in the same atomic step, and send only when the count is within the quota. Otherwise wait the time the script says the window has left. Gate every request through it: sdkgen’s system.fetch, a Kiota middleware handler, or an openapi-fetch middleware. A per-instance limiter such as the sdkgen ratelimit feature cannot hold a fleet to one quota.

You will need

Node 22 or later, a Redis that every worker can reach and that allows scripts, and one API key whose quota the workers share. Verified 2026-11-20 against Node 22.23.5. The gate runs no Redis: redis-standin.mjs implements the six commands used, with the replies the reference gives for SET, INCR, PEXPIRE, PTTL, TIME, and EVAL of the one script. The workers run on a virtual clock, so five seconds of traffic prints the same counts on every run. The sample is TypeScript only. A Go or Python worker can join the same window, since any Redis client can send the script, or use one of the libraries in the table.

Approaches compared

ApproachWhen it fitsWhat it costs youWhen to pick something else
bottleneck clusteringNode workers, and a limiter with queues, priorities, and a Redis datastoreA dependency that stores its own settings in Redis, shared by every limiter with that idWorkers in more than one language share the key
go-redis/redis_rateGo workers, and GCRA rather than fixed windowsA Lua script on Redis 3.2 or newer, and Go onlyYour Redis forbids scripts
Hand-written window in RedisAny language, and a limiter of about twenty lines that you can readA Redis that allows scripts, one call for every attempt, and up to twice the quota across a window edgeThe quota is enforced on a sliding window
Kiota middleware handlerA Kiota client, whose runtime runs every request through a handler chainThe chain lives in the runtime packages, so the limiter’s position in it is yours to chooseThe client is not generated by Kiota
openapi-typescript and openapi-fetchA typed fetch client whose onRequest middleware can waitNo retry of its own, so a 429 that slips through returns to youYou want retries in the same chain as the limiter
sdkgen ratelimit featureOne process, one client, and a quota of its ownA bucket per client instance, so twenty workers get twenty bucketsTwo or more processes share one key

Voxgig maintains sdkgen. It is one of six options here, not the recommendation.

A shared limiter is exact and adds Redis to every call. A per-process limiter needs nothing, but it either multiplies the quota or, divided by the worker count, wastes it when workers sit idle. The libraries trade a dependency for algorithms the hand-written window lacks.

Count the request and set the expiry in one script

sharedWindow is the whole limiter. It sends one script, and Redis runs a script atomically. INCR counts the request, and a counter with no expiry gets one before any other command can run. The reply is this request’s place in the window and the milliseconds the window has left.

export const WINDOW_SCRIPT = `
local n = redis.call('INCR', KEYS[1])
local left = redis.call('PTTL', KEYS[1])
if left < 0 then
  redis.call('PEXPIRE', KEYS[1], ARGV[1])
  left = tonumber(ARGV[1])
end
return {n, left}
`

export function sharedWindow(redis, { key, limit, windowMs, sleep }) {
  // PEXPIRE refuses a fraction after INCR has counted, and deletes the key on zero.
  if (!Number.isInteger(windowMs) || windowMs <= 0) throw new RangeError('windowMs must be a positive integer')
  return async function acquire() {
    for (;;) {
      const [n, left] = await redis.eval(WINDOW_SCRIPT, { keys: [key], arguments: [String(windowMs)] })
      if (n <= limit) return
      // A key lives through the millisecond it expires in, so wait one more.
      await sleep(left + 1)
    }
  }
}

No worker’s clock is read. Redis starts the window and Redis expires it, so a worker whose clock drifts waits the right time anyway. The script also repairs a counter it finds without an expiry, so one left behind by an older limiter lasts one more window.

Measure twenty workers against one quota

The sdkgen SDK was generated from api.json with the ratelimit feature added. Each worker gets its own client, and its system.fetch waits for sharedWindow before it sends. demo.mjs runs twenty busy workers and then two, against an API that answers 429 over 50 a second.

node demo.mjs
20 workers, one API key, quota 50 requests per second, 5 s of virtual time
  limiter (per worker)                      busy  requests per second        429s  Redis calls
  none                                        20  2000 2000 2000 2000 2000  9750  0
  sdkgen ratelimit, rate 50, burst 1          20  1000 1000 1000 1000 1000  4750  0
  sdkgen ratelimit, rate 2.5, burst 1         20  60 40 60 40 60              30  0
  sdkgen ratelimit, rate 2.5, default burst   20  80 60 40 60 40              50  0
  shared window in Redis                      20  50 50 50 50 50               0  350
  none                                         2  200 200 200 200 200        750  0
  sdkgen ratelimit, rate 50, burst 1           2  100 100 100 100 100        250  0
  sdkgen ratelimit, rate 2.5, burst 1          2  6 4 6 4 6                    0  0
  sdkgen ratelimit, rate 2.5, default burst    2  8 6 4 6 4                    0  0
  shared window in Redis                       2  50 50 50 50 50               0  260

The feature at rate: 50 holds each client to 50, and twenty clients send 1000. Its reference says the same: a per-client-instance budget, not a distributed one. Dividing the quota gives 2.5 each, which still overshoots. Twenty buckets that started together refill in step, so alternate seconds carry 60. The default burst equals rate, which lets 80 through in the first second, so the other rows set burst: 1. With two busy workers the feature sends 4 to 8 a second against a quota of 50.

The shared window sends exactly 50 a second either way. Twenty busy workers cost it 350 Redis calls for 250 requests. The hundred more are refusals, one for each worker in each window.

Wire the same window into Kiota and openapi-fetch

Kiota ran on api.json and wrote request builders with no HTTP code; its runtime package sends each request through a chain of middleware handlers. The documentation’s TypeScript example adds a handler at the front of the default chain. The sample puts it just before CustomFetchHandler, the one that sends, so the RetryHandler’s retries pass it too. kiota info 1.38.0 rates TypeScript at Preview maturity, and its runtime packages carry 1.0.0-preview versions.

  Kiota: ({ acquire, fetch }) => {
    // The default chain, with the limiter placed just before the handler that
    // sends, so a retry made by the RetryHandler waits for a slot too.
    const handlers = MiddlewareFactory.getDefaultMiddlewares(fetch)
    handlers.splice(handlers.findIndex((h) => h instanceof CustomFetchHandler), 0, new SharedLimitHandler(acquire))
    const adapter = new FetchRequestAdapter(new AnonymousAuthenticationProvider(), undefined, undefined, KiotaClientFactory.create(fetch, handlers))
    adapter.baseUrl = BASE
    const client = createBookshelfClient(adapter)
    return () => client.books.get()
  },
  'openapi-fetch': ({ acquire, fetch }) => {
    const client = createClient({ baseUrl: BASE, fetch })
    client.use({
      async onRequest({ request }) {
        await acquire()
        return request
      },
    })
    return () => client.GET('/books')
  },

openapi-typescript wrote types only, and openapi-fetch runs every onRequest middleware before it sends. An async one can wait for a slot.

node generators.mjs
20 workers per client, one shared window of 50 per second
  sdkgen         requests per second 50 50 50 50 50, 429s 0
  Kiota          requests per second 50 50 50 50 50, 429s 0
  openapi-fetch  requests per second 50 50 50 50 50, 429s 0

Kiota suits a document that also feeds .NET or Python clients, because its middleware page shows the same handler pattern for both. openapi-fetch suits a TypeScript service that wants one small dependency and no generated runtime to configure.

Check it worked

Seventeen tests run. The two that matter most assert [50, 50, 50, 50, 50] and no 429 for twenty workers, and the full quota for two.

node --test rate.test.mjs
1..17
# tests 17
# suites 0
# pass 17
# fail 0
# cancelled 0
# skipped 0
# todo 0
# duration_ms 2322.878846

Others pin the feature at 1000 a second for twenty instances, its overshoot at 2.5 each, each client behind the window, and the stand-in’s replies against the command reference. One sends a 503 with Retry-After: 0 through Kiota and asserts that both attempts waited for the limiter. Three hold the script to its promise. With a millisecond for every round trip the counter always has an expiry, a counter that has none gets one, and the stand-in refuses any other script.

When it goes wrong

The API returns 429 although every worker obeys the limiter. The window key is named from each worker’s clock, and clocks drift, so two windows are open at once. Name it from TIME, or let PX time it.

node pitfall.mjs
20 workers, quota 50 per second
  limiter                                      requests per second   429s
  window from worker clocks, half 500 ms fast  100 50 50 50 50        50
  window from Redis TIME                       50 50 50 50 50          0
  INCR then PEXPIRE, one worker dies           49 0 0 0 0              0

Every worker stops sending after the first second. A worker died between INCR and PEXPIRE, so the counter has no expiry and PTTL replies -1.1 Run both in one script, as sharedWindow does.

Opening the window with SET key 0 NX PX 1000 and then calling INCR leaves the same hole at the window’s edge. A key that expires between the two commands comes back from INCR with no expiry, and SET with NX can never replace it. One test moves a clock a millisecond for each command across that edge, and shows that no caller gets through afterwards.

The API still sees bursts over the quota in short spans. A fixed window permits its whole limit at the end of one window and again at the start of the next, so one test gets 100 through in two milliseconds. When the quota is a sliding window, use GCRA, as redis_rate does, or the sliding-window log that is pyrate-limiter’s default.2

When not to do this

Do not rely on the sdkgen ratelimit feature for a quota that two processes share. It keeps a token bucket per client instance. Use it for a single process, or behind the shared window as a local smoothing step.

Do not put Redis in front of a quota nobody enforces near the limit. The shared window adds Redis round trips to every request and a hard dependency on Redis staying up. Decide whether a Redis outage means fail open or stop.

Do not adopt Kiota or openapi-fetch for the limiter alone. Keep Kiota where its handler chain already carries your auth and retries in several languages, and openapi-fetch where one typed client and one middleware is all a TypeScript service needs.

Last verified

Verified 2026-11-20 against Node 22.23.5, @voxgig/create-sdkgen 0.31.9, @voxgig/sdkgen 4.44.3, Kiota 1.38.0 with @microsoft/kiota-bundle 1.0.0-preview.114, openapi-fetch 0.17.0, and openapi-typescript 7.13.0, and TypeScript 5.9.3. Every output block is what the command preceding it printed. Every Redis command in them went to redis-standin.mjs. compare-with-redis.mjs was run against Redis 7.0.15, where no reply differs over 42 commands, the script among them, and sharedWindow held 12 requests to 5 in every 200 ms. bottleneck, go-redis/redis_rate, and pyrate-limiter were not run; their options come from their READMEs.

Footnotes

  1. The INCR page carries this limiter as its own worked pattern, with ten requests a second per IP address. Its second version calls INCR and then EXPIRE, and the page notes the race at once. A client that performs the INCR but not the EXPIRE leaks the key until the same IP address is seen again. The page fixes it with a Lua script sent with EVAL, or with a list. Its first version names each key by the client’s current Unix time, the clock-named window that pitfall.mjs shows overshooting when clocks drift. ↩︎ Back to text

  2. redis_rate describes itself as GCRA, also known as leaky bucket, based on rwz/redis-gcra, and needs Redis 3.2 for replicated script commands. pyrate-limiter’s README adds that GCRA state shared between machines needs a shared clock, and that its RedisBucket keeps items in a sorted set scored by timestamp. ↩︎ Back to text

Read this page as markdown · All how-to guides

Generate the client instead of writing it#

Retries, timeouts, pagination and auth are the same problems in every client. Voxgig generates them from your OpenAPI description, in 23 languages, from one model.

Listen to Fireside, the Voxgig podcast

Since 2018, Richard Rodger has talked with developer advocates, event organizers, and the people who build developer-facing companies about the specifics of the work.